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CHAPTER I 

IMPROVING THE MANAGDIENT OF PUBLIC SCHOOL SYSTE>fS: 
A TEST OF SIGNALS OF RELATIVE SCHOOL PERFORI'lANCE 

A. INTRODUCTION 

This report pri^ssnts the results of a field test conducted by the 
Program Evaluation staff of The Urban Institute in the Atlanta Public School 
System during the 1972-73 school year. The purpose of the test was to deter- 
mine the effects on school system management, decisions, and operations of 
introducing into a school system specially prepared information on the rela- 
tive achievement levels of schools and grades serving students. of siTnllay 
economic levels. 

Student performance has become- an issue in many large urban school 
systems. In Atlanta, for example, community groups have sought and obtained 
information on the achievement levels of the students in individual schools. 
School systems have tried a variety of approaches in attempts to improve 
student performance, but school administrators typically do not know how 
successful their own efforts or the efforts of others have been. The fed- 
eral government has required evaluation of individual projects and in so 
doing has added to local interest in student performance, but information 
from federally required evaluations has not generally played an lonportant 
role in school system planning or administration* 

Officials with major management responsibilities in large school 
systems often do not know precis«2ly where to look for success or failure* 
Present indicators of success cr failure in the classroom grow progressively 
weaker as distance from the classroom increases. Part of the problem stems 
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from the absence of clearly stated goals and objectives for education and 
from the absence of agreed-upon measures of success in meeting those goals.. 
Even in the limited area of a school system* s success in teaching basic 
skills, however, the parties f:o a <]iscussicn of success or failure rarely 
use the same set of assumptions and measurements in identifying success or 
failure, and the data used in any discussion are rarely current • 

In a large school system, the duties of management above the level of 
the schonl principal include many support functioni:- not directly tied to 
the performance of the students* Support functions such as payroll, facil- 
ities, equipment, personnel functions, ensuring coir.;)liance with state anu 
federal laws, and purchasing offer economies of scale in a large syste'^^ and 
are generally performed by centralized units. 

Our examinations of large school systems have found Ci-^anized manage- 
ment- in the support area, but no we^-l-def ined management structure aimed at 
systematically improving student performance. The setting of minimum 
acceptable levels of performance and assurance of quality in the areas of 
instruction and curriculum are generally accepted as system-wide functions. 
Management tools for applying even these limited standards, however, arti 
rarely available. The reason generally given for this lack is the diffi-- 
culty of creating performance indicators that are current, readily inter- 
pretable, accessible to decision makers, and yet specific enough to be the 
ba3is for decisions and actions aimed at improved performance. 

The Urban Institute has continued to explore the question of what would 
constitute school system management approaches aimed at Improving perf ormancti. 
At the same time, an early effort was made to remedy the lack of easily usable 
perfomance information. This was done in order to test the extent to which 
lack of infurmc^.tion itself was a problem holding back improved performance 
Q in large school systems, 
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Our exploratory effort into these management questions was initiated 
in" January 1971, usirig Urban Institute research funds, and expanded in 
January 1972, v^hen support from the Office of Economic Opportunity was 
obtaln'^a. Phase I of that effort (ending in October 1972) produced usable 
i^erformance information.* The technique used existing school system data 
on student achievement and participation in the subsidized school lunch 
program to locate specific grades in individual elementary schools in which 
the level of average student performance was significantly better or worse 
than the level of performance in schools of similar economic level. 

The Phase II work reported here (November 1972 to October 1973) was 
designed to make information on relative school performance widely available 
In a school system and to observe the influence of this information on 
decision making in areas related to student performance. Signals of rela- 
tive school performance covering school years 1970-71 and 1971-72 were 
created and introduced irto the school system in the f all , and early winter 
of 1972, and a detailed study was made of their impact. 

Two results were expected from this effort. First, a test would be 
made to determine if the simple provision ^f performance information to an 
operating school system produces different managemert decisions. Second, 
iu the process of that test, better inf ormat *.on would be acquired on how 
the school system managemenc aci;ually attempts to influence performance. 

Since January 1971, staff from The Urban Institute and the Atlanta 
Public School System have worked together to define and shape various 
aspects ox thit project, including the work reported here. Although infor-- 
ination from other school systems and other research results have also been 

* See Bayla F. IsTiite et . al . , The Atlanta "Project; Developing 
Signals of Relari' i School Performance , The Urban Institute, October 1972. 



of value, it was decided early in our research efforts to use an actual 
large school system as -a ''laboratory" in -order to force the development 
effort to deal with reality. The Urban Institute staff chose Atlanta from 
among several large urban school systems because it tested all students 
twice yearly, had sufficient data available to explore a variety of perform- 
ance questions, maintained a strong system-wide interest in improving per- 
formance, and had personnel who would fully support the basic aims of the 
project. This cooperation has never wavered, and it has been largely due 
t-^ "his relationship that we are now in a position to discuss large urban 
c:chool systems as they actually function. 

B. THE URBAN INSTITUTE APPROACH 

School administrators interested in performance often become overwhelmed 
by numbers. All too often, the response of researchers has been to create new 
and different numbers. The approach used here was to make comparisons of the 
relative performance of similar students, isolate significant cases of extreme 
performance, and display the results as a series of charts in which red "signals' 
denote levels of relatively low performance and blue "signals" denote levels 
of relatively high performance. This signaling technique permits compact 
displays of information on the relative performance of every grade in a school 
for several years and for different 'subject areas or for every grade in every 
school in an administrative area for successive years. In this way, a large 
amount of information of high interest to the school system management can be 
extracted from the existing data and provided in a compact, accessible form. 

* See page 11-7 ff. 
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Even though there is much argument about the applicability and accuracy 
of achievement tests, there is general agreement that students in elementary 
schools should learn to read and to solve simple arithmetic problems. By 
using relative compfirisons and by being conservative in setting boundaries 
for what is to be considered "extreme" performance, this approach provides an 
answer to a question of great interest to school system administrators: "Where 
exactly are our successes and our failures?" 

The following hypothesis was then tested by the Urban Institute in Atlanta 

in the past year: 

Information on relative performance, when introduced into 
a sufficiently well-understood planning and management structure, 
does (or does not) produce measurable changes (a) in the decision- 
making process of a local school system, (b) in the decisions made 
by school officials, and (c) ultimately, in school performance. 

In its simplest form, the hypothesis might be viewed as a test of whether 
the provision of performance information alone is sufficient to produce pur- 
poseful systemwide changes in management in the area of performance i Such 
information is certainly a necessary condition, for without usable performance 
information, decisions cannot be based on the results of past actions. The 
presence of good performance information alone, however, may not be suificient 
to cause a large school system to change its decision-making process. To 
test the effects of relative school performance information on a large school 
system, compact performance information was made available, and a detailed 
study was made of the impact of the information upon system operations. 

This year of study focused on those portions of the school system which 
have some immediate, direct impact on the classroom. Based on the earlier 
work in Atlanta, it had become apparent that the introduction of signal in- 
formation would not have an equal impact on the operation of all parts of the 
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school system.' Activities were selected for study on the basis of two criteria. 
First, there must be some direct interaction between the activity and the 
classrooms of the school system. Secondly, the activity must have some flexi- 
bility and discretion that could be used in reacting to signal informavxion 
in the short run. The three activities identified by Urban Institute staff 
and our advisory board in Atlanta that met these criteria were the following; 

• recruitment, assignment, and reassignment of . teaching staff , 

• the instructional program, and 

• improving the skills of classroom teachers. 

The study attempted to learn how these activities are normally conducted in 
Atlanta, what impact the introduction of signals had on these activities, and 
the resulting effect upon performance in classrooms. Research into thfjse ' 
activities was based on extensive structured interviewing, surveys, formal 
reporting by Atlanta Public School personnel, and attendance by Urban Institute 
staff at numerous operational and staff meetings in Atlanta- 

In considering the information presented below, it is important for the 
reader to keep in mind two facts about the activities presently occurring within 
the school system: 

• Actions taken to improve performance must somehow do so 
through their effect on the education taking place in the 
classroom. In the material below, each management activity 
is examined in terms of the impact of the activity on 
classrooms and in terms of the use or non-use of informa- 
tion about relative school performance. 

• ^ We do not now know the -f^xtent to which these activities 

should rely on performarice information in conducting 
their work. We are not yet in a position to delineate 
where the use of performance information should begin 
and end or what management approaches aimed at improved 
student performance should be. 
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C. SCHOOL SYST2>I MANAGEMENT AND PRACTICES 
^. BEFORE THE INTRODUCTION OF SIGNALS 

This section summarizes the operational characteristics of the school 
system before the introduction of signals. A goal of The Urban Institute 
research effort has been to unde-^ stand £^nd improve the management of Large 
urban school systems. It is important, in order to do this, to understand 
how school systems are managed at preseu..* Not only have the Atlanta Public 
Schools provided an excellent source of data for developing the signals, 
it has also provided The Urban Institute with a unique opportunity to 
observe a school system in its detailed operation and to examine how manage- 
ment decisions are made ar-^ implemented. Atlanta Public Schools staff mem- 
bers have fully cooperated in providing information on their decision making 
and in facilitating --ar collection of operational data. Independent of our 
findings on the effects of signals on the operation of the school system, 
several factors have been Identified concerning the management of school 
systems which appear to have significance for Atlanta and other large urban 
school systems. 

Atlanta, like most large school systems, operates through a line manage- 
ment chain that starts with the classroom teacher. The chain goes up through 
school principals to an Area Superintendent^ , and on to the Superintendent and 
for schools in a given geographic area, and on to. the Superintendent and 
his immediate staff, who report to an elected School Board. Specialized 
support functions such as personnel, finance, instructional services, 
research and development, and buildings and facilities are handled by sep- 
arate staff divisions headed by Assistant Superintendents who report 
directly to the Superintendent. Area Superintendents are responsible for 
operation of the school system within their geographic Area. In Atlanta, 

ERLC 
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the Area Office has a support staff composed primarily of Resource Teachers. 

These Resource TeachersL can be characterized generally as experienced 

teachers who provide specialized assistance to the principals and teachers 

in individual schools. The school system is inanaged through a direct line 

organization that is supported by a fairly large and complex bureaucracy 

with distinct organizational lines and some overlapping functions. 

General observations made on school system operation in the period 

before signals were introduced include the following: 

# Information on the performance of students was, in general, 
neither available to nor used by management units other than 
the classroom teacher, principals and Resource Teachers.* 
Users who did use some performance data for the most part 
dealt with absolute performance levels, rather than with 
information on variations in performance among schools and 
grades. serving similar children. 

^ In general, mechanisms were lacking for activities of the 
central school system management to tailor their activities 
to actual variations in performance among or within schoo'cS* 

There have been attempts by variour. activitie.3 to apply their resources 

S'"'',ectively to particular grades and schools; however, in nearly every case 

« 

examined, either the thrust of the activity was diminished or its direction 
was altered as the activity filtered down to the classroom level. 

Several more specific results have been reported to support these 
general conclusions. VThile these specific results may seem unique to Atlanta, 
evidence from other cities indicates that the general results above are 
applicable to other large urban school systems as well: 



* An exception to this was the Comprehensive Instructional Program 
(CIP) which had created a system for supplying teachers directly vith 
information about their students. CIP has also used achievement data in 
attempts to evaluate its own efforts. 

Atlanta's Research and Development Department has generated and 
distributed absolute performance information and relative gains during the 
course of this study. This information, which is complementary to signals, 
was also distributed in 1972-73. 
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— New teachers have been hired and initially assigned without 
any attempts to use the specific performance record of the 
group of students eventually taught. 

--Principals have no advance information about the character- 
istics of teachers new to the school (e.g,, information on 
the teachers' instructional strengths and weaknesses). 

— Principals have little input . into decisions about which 

teachers will be assigned to their schools; those making 
■ the decision do not have available student performance 
iiiformation in a form that could be useful in making the 
decision. 

— The major factor which has governed staffing assignments 
is the need to meet the requirements of the court-ordered 
racial ratios in individual schools. Within these con- 
straints, neither the information nor the management 
mechanisms needed to take relative student performance , 
information into consideration was evident. 

In the instructional area: 

— The Instruction Division prepares the same curriculum 
guides for use in all schools. Once the guides have . 
been circulated among schools, little or no information * 
on the implementation of the curriculum reaches the 
Instruction Division. No formal mechanism exists to 
assess good practices among schools or faults in the 
curriculum. 

— The Comprehensive Instructional Program provides teachers 
with assessments of the progress of their students in 
developing basic skills. This program takes diagnostic 
measures from the individual student and returns the 
information directly to the teacher. It is the best 
example in Atlanta of direct use of performance informa- 
tion. In this case the performance information is 
absolute, not relative to the school population. No 
equivalent stream of data exists for the central staff 
departments in relation to their own work* 

— ?Each Area was assigned at least a few Resource Teachers 
whose area of specialization was reading. No assessment 
has been made of the effect those Resource Teachers have had 
on student performance. 
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— A textbook coinmittee approves texts for use in the 

classroom, and the Textbook Coordinator on the Instruction 
Division staff advises individual principals on choosing 
textbooks.^ Both the textbook committee and the Textbook 
Coordinator operate without empirical information on variations 
in performance of Atlanta students using the various texts. 

— Directors of special programs (e.g.. Title I) design 
programs and make changes in programs generally using 
little information about the effects on the performance 
of students in the schools in which the programs are 
implemented, even though some information of this type 
is generated. 

• In the staff improvement area: 

— Although Resource Teachers make decisions on which 
grades (and sometimes which schools) to visit, these 
decisions are seldom based on relative student achieve- 
ment information. 

— No central records have been kept of the university cour.c^es. 
Instruction Division courses or Area in-service programs 
that teachers attend. The Personnel Division keeps some 
records for salary purposes. The effect of such training 
on student performance is unknown. 

— Neither principals nor Area Superintendents are systemat- 
ically Informed of the teachers who complete coifrses. 
This has made intuitive evaluations by school system 
" personnel of the effects of such courses impossible. 

— Although each Area conducts many workshops each year, 
most of the subjects are no directed at basic skills. 
Proficie;.cy modules, a newl. introduced teacher training 
program, are available to all teachers. Although suc- 
cessful completion of the module program entitles a 
teacher to a salary increment, no link has been made 
to determine what effect completion of the modules has 
on student performance. 

— A rating instrument designed to test classroom teachers' 
proficiency in the teaching of reading has recently been 
introduced. This instrument has not been validated to 
determine if its ratings correlate with the teacher's 
ability to improve the performa.ice of her students. 

— Classroom teachers and Resource Teachers seem to have . ^ 
little or no input into decisions made by the Instruction 
Division about in-service training offered. 

Neither the Instruction Division nor the Area Offices use 
student performance as a measure of the effectiveness of 
the courses or programs they offer to teachers. 
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These findings . summarize the situation with respect to the avail- 
ability and use of relative performance information. The findings do not 
represent an evaluation of the management activities of the Atlanta School 
System or any other system, nor has the feasibility of evaluating manage- 
ment activities through the use of relative student performance in the 
schools been established, WTiat this preliminary work does represent is 
a survey to determine who might be interested in relative performance 
information as well as an attempt to establish how management efforts are 
presently organized. Our information to date- indicates that management 
activities as described here are conducted in a similar manner in many 
school systems. Given this management environment, one could expect 
that personnel in a school system might respond favorably to the concept 
of signals displaying performance information — as they did. Most are 
committed to improving performance. However, one can also begin to 
anticipate that the chance of an effective management response from a 
school system^s staff and line manag,ement elements to performance ' inform- 
ation might be low — as, in fact, it turned out to be. These results are 
discussed in the next section. 

D. THE EFFECT OF INTRODUCING SIGNALS OF RELATIVE PERFORMANCE 
ON THE OPERATION OF A LARGE URBAN SCHOOL SYSTEM 

The major thrust of the past year's study was to make relative per- 
formance information available to managers and administrators of the school 
system and to determine what effect the availability of the information had 
on the operation of tue school system. Atlanta school officials cooperated 
wholeheartedly in the -distribution and e*xplanation of the signals and in 
f^cilitating our collection of information to document the effects observed. 
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Four general results which characterize the findings of this part of the 
study are the following: 

• School officials readily accepted the signals and often had 
high words of praise for the signa].9 as a useful and under- 
standable mechanism which they could use in their manage- 
ment roles to diagnose performance in the school system and 
plan program activities. 

• In only a few cases did these same officials give evidence 
of actual use of relative performance information in making 
significant management decisions. 

• There was little evidence of a shift of specialized school 
system activities to grade levels in schools which were 
signaled as extremely high or low in relative performance. 

o In the few cases, where activity did seem to be dispropor- 
tionately allocated to grades having a particular type of 
signal, such as in a particular Area or with a particular 
Resource Teacher, there were converse allocations of effort 
which demonstrated that the signals were not- having a con- 
sistent effect. 

These general results are drawn from specific observations that were made 
throughout the school system on how and what decisions were made, how and to 
what extent they were implemented and what, if any, effect they had on 
classroom actJ.vities. Some more specific results of 'our analyses in the 
three areas on which we conceatrated our attention are listed below: 



Personnel : 

— The Personnel Division staff showed an interest lii using 
signals to improve procedures for screening new applicants, 
and reassigning current sfaff. There is, howevar, no evidence 
that anyone followed through on this interest and developed 
new screening or assignment techniques based on the signals. 

— Since no new teachers were hired, signals could not play a 
role in that activity this year. 

— Area Superintendents, who have authority over all transfers, 
in teaching assignments in their Area, generally did not 
use signals in making these decisions. 
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— One Area Superintendent did reassign several teachers who had 
been in a school with several red signals to schools with 
several blue signals in the hope that the teachers would 
be influenced by the high performcnce of teachers in their 
new schools. These assignments became effective in 
September of 1973. 

Instructional Program: 

— In general, no evidence was detected of cases where signals 
were considered in decisions involving the shape of the 
instructional program for the school system. 

— More particularly, neither performance information in general, 
nor the signals in particular, seem to have affected the choice 
of pilot schools for the new elementary school curriculum or 
the appraisal of the program as it is being implemented. 

— The Instruction Division staff responsible for the Com- 
prehensive Instructional Program (CIP) , a program started 
three years ago to provide intensive assistance to, teachers 
and students in teaching reading and arithmetic, were 
familiar with signals but gave no indication of having used 
signals in the operation of the program. (In the past CIP 
activity was evaluated with achievement test data, but this 
has h ien discontinued.^ 

— Although several principals reported making decisions on the 
adoption of new textbooks this year for the coming years, 
signals played no part in the decisions about particular tejirts 
or materials. 

— In general, neither Area Superintendents, Curriculum Coor- 
dinators nor principals reported using signals or other 
performance information in discussions about the instructional 
programs — either in general or for specific schools or grades. 
In a few cases there was evidence of a principal using signals 
to support decisions on restructuring a particular instructional 
program. 

Improving the Skills of the Teaching Staff: 

— Two mechanisms for improving teacher skills were examined 
in depth: in-service tr^/ining and direct assistance to 
classroom teachers through the Area Office Resource Teachers • 
In neither case is there evidence to support a conclusion 
that signals had a general, systematic and consistent effect 
on decisions concerning these activities or on the pattern of 
activities that resuJted. 
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— In particular, except for one or two principals, no Atlanta 
school official reported using information from the signals 
in designing in-service programs or in determining who should 
attend thela. 

— When the school system is considered as a wholes 

distribution of Resource Teacher activity to grades receiving 
signals indicating high, low or average relative performance 
is nearly identical to tiie proportion of grades ha\ ing the 
various types of signals* That is, the observed distribution 
of assistance showed no tendency to disproportionately favor 
grades signaled as having a record of extreme relative per- 
formance — either high or low. 

— When the distributions of Resource Teacher assistance are 
considered on an Area Office basis, statistically significant 
cases of disproportionate assistance being given to grades 
having a particular type of signal are detected. Only in 
Area III, however, were the differences large enough to have 
any operational signif icance* 

— In those grades visited by Resource Teachers, grades having 
a signal associated with low relative performance did tend 
to receive a bit more assistance in terms of time spent 
providing the assistance and of the number of occasions on 
which assistance was provided, but the differences appear 
to be due to a few Resource Teachers and a few grades, and 
the Overall differences are not statistically or operationally 
significant. 

— Thus, while there are instapces where Resource Teachers 
were users of signals and disproportionately provided 
assistance to grades having a particular typa of signal, 
there is no evidence of a systematic or pervasive tailoring 
of the delivery of Resource Teacher assistance in response 
to the signals of relative performance. 



The results of our analyses are discussed in greater detail in 
Chapter V of this report. Based on these results, several implications 
can be drawn on the management of large urban school systems and the value 
of performance information in the operation of them. These implicationj 
are addressed in the next section. 
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E. IMPLICATIONS OF WHAT TO IIAVE LEARNED 



Our work in and with the Atlanta School System has several implications 
for the operation of large urban school systems: 

• Decisions made at levels above the school that concern the 
operation of the schools are generally made without infor- 
mation regarding performance at the classroom level and 

the variation that exists at this level, whether this infor- 
mation is available or not. Furthermore, there do not 
appear to be well defined management mechanisms or arrange- 
ments which can be ut:<lized by staff or line units in 
responding to relative performance Information. 

• In the past, relative perfomance information was not 
available in an immediately usable or understandable form 
for use by school system decision makers. [Some use had 
been made of abrolute performance data.] 

• The Urban Institute has provided relative performance infor- 
mation which school officials accepted and praised; however, 
this information was not generally used in making operational 
decisions nor did it have a consistent or obvious effect on 
the activities of the system. Further, management mechanisms 
for making use of performance information in decisions 
Intended to affect performance in the schools do not exist or 
are at cross purposes \iith each other. 

• Further research is necessary to say in whl,ch parts of a 
school system the use of performance information is actually 
feasible and to design the management techniques for bringing 
such information to bear on decisions in a systematic way. 

Several further implications can be drawn: 



• Since signals were praised and accepted by most school 
officials and actually utilized In a few instances, the 
continued publication of signals on an in-house basis may, 
through time, result in the movement towards a management 
system based on performance Information. This is a viable 
possibility since the costs of production are relatively low. 

• At a mininvum, however, it would seem necessary at least to 
develop a unified approach for various components of the 
system to use either the signals or ether relative perfor- 
mance information. 



e In order to have some degree of confidence in achieving 

better performance management capabilities for large urban 
school systems, it app<^ars necessary to adopt a two-pronged 
approach. 
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— iDiprovement In the performance measurement systems. The 
currant signaling system has obvious linitations for 
central staff use which could be overcome through alter- 
iiative approaches to organizing student achievement scores. 

--Development of management techniques for school sv^tem 
use of relative performance inf ormatioxi. 

F. SIGNA^.S AITD SIGNALING 

As has been described above (and in more detail in Chapter II) , "signals" 
were used as a mechanism to display the relative performance of individual 
grades in each elementary school. The mass of achievement score data was 
reduced to five levels of performance in response to problems of reliability, 
validity and understandability pointed out by school system personnel in 
comments on the use of student performance- data. 

The past yearns work has demonstrated that school systems could produce 
signals in-house without large expenditures of resources, if achievement 
test data and indication of the socioeconomic level of Included schools or 
grades are available in a machine-processable form. Documentation for com- 
puter programs that have been used to produce the signals and a statistical 
analysis of the data is provided in the Appendix to this report. It is 
estimated that Altanta, or any other similar sized school system having com- 
parable data, could produce the signals in about two to three weeks following 
the grading of the tests at a cost on the order of -^$3, 000 (excluding printing 
costs). First year costs may be slightly higher, but on-going operational 
costs should be no more than this amount. This cost is that for starting from 
raw student scores and producing signals. 

During the last year, Atlanta has modified its computer processing 
system and now can produce grade level averages for each school. A small 
change in their subsidized school lunch computer program would allow the 
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calculation of the percent of students receiving subsidized school lunches on 
the basis th^t has been used in this system. With this information already 
• available, only the signal preparation programs need be run. Our present 
program typically can produce signals from this information with one day of 
analyst tine and perkaps $100-$200 of completer time. 

The analyses conducted on the signals this year have also demonstrated- 

that: 

• The technique used to identify schools performing high or 
low relative to schools Laving students from similar socio- 
economic conditions can indicate extremes of performance 
acros'3 the full range of socioeconomic conditions existing 
in the Atlanta stud ant population. 

• Grades that are signaled actually do represent extremes of 
performance. Individual test scores from grades that were 
signaled were examined in detail 'to verify that the variation 
observed for the grade was not just a random event that could 
statistically be expected to occur in the number of cases 
detected. In fact, signaled differences are generally statis- 
tically significant and cannot be attributed to chance 
variation. 

• Within their error range, the signals appear to be able to 

. identify a recurring phenomenon. The probability of a grade 
having a certain signal in one year is dependent on thf3 sig- 
nal it received during the previous year. These conditional 
probabilities appear to exhibit some stability through time. 
There were few significant differerces in the transition 
probabilities between academic years 1970-71 and 1971-72 and 
between the years 1971-72 and 1972-73. If a school system 
can detect and react to such phenomena, then it has the 
potential for taking actions aimed at affecting these tran- 
sitions and achieving higher student performance. 

VThile the analyses during this phase of the study have confirmed the* 

technical soundness and usability of the signals, they have not attempted 

to fully exploit the relative performance information that can be extracted 

from data collected by the school system. Further work is also needed to 

establish the feasibility of managing to improve performance. Such analyses 

were not the principle foci of this phase* of the study; the majority of our 



4 



1-18 



effort was expended on analyzing the existing mx^nagement of a large school 
system and the effect of the Introduction of performance information in the 
form of signals. Questions of what types of management organizations ire 
possible and to what extent each possible organization could be supported 
with accurate relative performance information are the logical next steps 
of this effort. Sufficient infoirmation has already been collected to support 
most of this work. 



G. ORGANIZATION OP THE REPORT 



The remainder of this report is divided into four chapters. The 
reasons underlying the development of signals of relative performance, the 
techniques used in that development, and properties of the signals are 
described in Chapter II. Chapter III explains the general setting in which 
the' study took place and the procedures used in its implementation. In 
Chapter IV, the operation of the Atlanta School System is discussed and an 
examination of the management of Activities chosen for investigation is pre- 
sented. Chapter V presents the results of those investigations. 



ERIC 



CHAPTER II 
SIGNALS OF RELATI^/E PERFORMANCE 



While the re.iaining chapters of this report describe the impact of new 
information on the management of a large school system, this chapter focuses 
on how and why that new information was developed. The chapter begins with 
a discussion of the phi.osophy underlying the focus on relative performance. 
The technique for generating signals of relative performance is explained. 
Some of the properties of signals are analy2ed in the final section^ 

A. ASSUMPTIONS UNDERLYING THE DEVELOPMENT 
OF SIGNALS OF RELATIVE PERTOKMANCE^ 

The ultimate aim of this project is to improve the capacity of a large 
school system to plan and manage its activities in a manner which will make 
better use of available, limited resources, "^he technique proposed provides 
school officials with information about the relative performance of students 
in schools and grades within those schools throughout the school system. 
Tue information on relative school or grade performance is derived from 
existing data assembled in a concise, simple display to signal how well or 
badly a school is teaching basic skills. 

Why produce signals at all? Of what value are such signals? The de- 
velopment of signals of relative performance has been guided by the belief 
that school officials need concise, current information in making decisions 



1- Sections A and B of this chapter are taken largely from Chapter I 
o^ The Atlanta Project; Developing Signals of Relative Performance, Bayla 
F. White, et> al .. The Urban Institute, Washington, D.C. 1972. 
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about either school system policy or day-to-day administration. Every school 
administrator, principal and teacher ic overwhelmed with numbers. The prob- * 
lem has been how to convert those numbers into meaningful indicators of what 
is happening in a large, complex gchool system. Too often, the researcher's 
response has been to create more and different ntimbers. But data collection 
in a large school system can be a time-consuming, expensive operation. Unless 
these new numbers are pertinent to the needs of educational decision-makers, 
there is no reason to collect the data at all. Therefore, the signals devised 
in this project were built on existing data, but assembled and displayed in 
a new format. 

One of the most important operating assumptions behind the development 
of the signals has been that officials in large school systems do not know 
precisely where to look for success or failure. Part of the problem stems 
from the absence of clearly stated goals and objectives for. education, and 
agreed upon measures of success in meeting those goals. But even in the 
limited case of a school system's success in teaching basic skills, school 
officials rarely agree on where to look for specif ic* successes or failures. 
Data used in any discussion are rarely current. Not every party to a dis- 
cussion of success or failure uses the same set of assumptions in identifying 
success or failure. Clearly, in order to know where to look for success or 
failure in teaching basic skills, there is a need for a single standard of 
measurement based on the most current data possible. 

The authors of this report believe that the measurement should relate 
to student performance j for it is data on performance that have been missing 
elements in the management of public school systems. It is technically pos- 
sible to develop a signaling system that would tell school officials which 
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school libraries have the most books or which jschools have the highest pupil/ 
teacher ratios. But, unless these signals are related to how well the stu- 
dents are performing, it is impossible to assess whether more books or more 
tenured teachers or fewer pupils per teacher are desirable. Therefore, the 
signaling system developed in this study relies on student performance as 
its dependent variable. Achievement test data are presently the most readily 
and universally available indicators of student performance in the area of 
basic skills development. Many problems exist with the interpretation of 
achievement test results. This signaling system has, however, been designed 
to reduce the effects of these problems. 

Since the signals are built on achievement test data, what should 
be the appropriate unit of observation? The signaling system focuses on 
performance in each grade within a school as the reporting unit, rather than 
the individual student or groups of students in particular projects. The 
school — which is composed of a set of grades — is the basic administrative 
unit of the school system. Although within a school the staff attempts to 
deal with the needs of individual students, management decisions about the 
allocation of educational r^^.sources inevitably involve the cchool or the 
grade within a school as their lowest common denominators. 

That signals should indicate relative performance is perhaps the single 
most important assumption underlying their development. A school system has 
little control over the background characteristics of its ' students . Students 
are the raw material of the educational process. A group of students enters 
school in the fall ax^d leaves that grade, or perhaps that school, in the 
spring. It is the job of the school system to structure its educational 
programs to the characteri itlcs and needs of its students. Educational 
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research has shown that a strong correlation exists between the socioeconomic 
level of students and their performance on achievement tests, which is used 
as the dependent variable in generating signals. 

Given these facts, it was deemed essential that the signals take into 
account the characteristics of students in the school system and compare 
relative performance of schools with similar student populations. Thus, 
comparisons are not made between the very rich and the very poor.. Rather, 
signals are based on comparisons of similar schools, in which performance 
is expected, a priori , to be similar. Signals based on extremes of perform- 
ance in similar schools provide -school officials with a much more precise, 
meaningful technique for pinpointing problems and successes in a large, 
diverse school system. Once problems or successes based on relative perform- 
ance' are located, then the educator can make more informed decisions about 
what action should be taken in a given situation. 

Finally, because the signals were devised as a management tool, the 
information they provide is intended primarily for the use of school admin- 
istrators, from the principals up through the administrative structure of 
the school system. Although the classroom teacher should also be interested 
in relative performance, it was felt that signal i.if ormation should be 
directed primarily toward other levels of school administration, specif- 
ically uhe principals and central office staffs. The classroom teacher is 
closest to the process of education and, as the primary dispenser of 
"education," already has information about how successful the process is. 
Present indicators of success or failure in the classroom grow progressively 
weaker as the distance from the classroom increases. Decisions which 
directly affect the process of education as it occurs in the classroom are 
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often maHe by personnel several levels removed- 'from the level where instruction 
takes place. Signals provide a device which enables busy school officials to 
• have a better idea of what is happening in the classroom, 

B. THE BENEFITS OF A SYSTEM miCH SIGNALS 
EXTREMES OF RELATIVE PERFORMANCE 

The potential benefits of information which signals extremes of perform- 
ance are implicit in the aL^sumptions which underlie its development. Any 
school system has limited resources, including money, personnel, and time, 
to be used in providing education to its students, and potentially limitless 
demands on those resources. The pivotal management question is how to make 
maximum use of those resources. By giving school officials clues about what 
is happening in the schools and v;hf-re, school administrators will have an 
important tool to use in making informed decisions about how and where to 
allocate those scarce resources. 

While signals can be used to locate extremes of 'performance , by them- 
selves they do not address the question of why the performance level exists 
or of what should be done about it. However, the information that signals 
provide can be used by school officials to diagnose problems and to prescribe 
appropriate treatments. Ideas for treating problems abound. There is a 
tendency to apply a particular remedy in shotgun fashion. A system which 
identifies schools with similar student populations can be- extremely useful 
in deciding v/here to apply a remedy and in evaluating its success. It is 
important to remember that signals call attention to success as well as fail- 
ure. Unfortunately, educators know as little about what causes success as 
they 3o about what accounts for failure. Attempts to account for apparently 
successful schools or grades can help school officials narrow the range of 
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possible remedies for problems, thereby enhancing the likelihood of success 
for the remedy chosen. - 

The signals flag only extremes of relative perf j^nnance. The definition 
of what constitutes extreme performance has been established so as to elimi- 
nate many of the arguments about the accuracy of achievement tests in measur-- 
ing individaal student achievement. The signals, by design, flag only those 
cases in which relative performance is extremely high or low. Consequently, 
the numbers of signaled cases are more compatible with the limited resources 
available to affect those cases. 

Information from the signals is potentially most useful in making deci- 
sions which have a direct impact on the classroom or grade because it is 
from the grade -that the signals derive. The pattern of signals from a partic- 
ular grade and subject area, such as 4th-grade reading, can provide important 
clues for curriculum development activities in a school system. The pattern 
of performance in a particular school can be used in making decisions con- 
cerning staffing needs for that school. Patterns in achievement in particular 
types of schools can be used to identify teacher training needs or to channel 
teachers to in-service training opportunities. Thus, in the short run, 
signals of extremes of relative performance can be an important ingredient 
in decisions about how to target flexible resources within the school system. 

In the longer run, the signals and the curves can be used to focus 
attention on important policy questions which confront school boards and 
top decision-makers within a school system. The signaling system does not 
detect net changes over time in the performance of the Atlanta School System 
as a whole. It provides a tool for use in identifying individual situations 
that produce extremes in performance in selected grades. The shape and level 
of the fitted cur/es, however, raise the questions of overall system perform- 
ance and of which types of schools to help. One policy response might be to 
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concentrate efforts on improving the performance of every school in which 
relative performance is extremely low. This might mean putting effort into 
a rich school where performance is at or above the national norm, but below 
the performance level of similar schools in the system. Or, a policy decision 
might be made to focus attention on the schools where absolute performance is 
lowest, regardless of the fact that some of those schools may have high level j 
of relative performance. Signals do not tell education policy makers which 
course of action is better. But the signaling system will help frame poli.':y 
issues more clearly, and thus should raise the level of debate about the 
direction of public education. 

• 

C. THE DERIVATION OF SIGNALS 
OF RELATIVE PERPOR^IANCE 

The information — signals— distributed to Atlanta officials has been 
designed to enable school personnel and administrators to identify grades at 
individual schools where the students* performance is cousiderably better or 
worse than might be expected of students in similar schooJs. Performance is 
measured by mean (average) level on the achievement test batteries adminis- 
tered to Atlanta students in the spring of each year. Schools are identified 
as similar based on the level of student participation in the free and reduced- 
priced lunch program. 

In April of each year, Atlanta administers a battery of standardized 
achieveLient tests to every student in Grades 1-7. Data on mean (average) 
achievement for the three years 1971, 1972 and 1973 have been used in the 
production of signals. For the years 1971 and 1972, signals were based on 
results of the Metropolitan Achievement Test batteries; for the year 1973, 
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signals are based on results of the Iowa Test of Basic Skills. After coa- 

sultation with Atlanta officials, only results from the reading and arithme- 

2 

tic problem solving subtests were used in the initial production of signals. 
The signals identify extremes uf performance relativ<2 to the economic 

composition of a school, as measured by participation in the free and reduced- 
price lunch program. Although educational research indicates that the socio- 
economic background of students accounts for much of the observed variation 
in achievements most school systems, including Atlanta, do not keep accurate 
or current data on the socioeconomic status of their students. However, 
participation in the federally subsidized school lunch program can be used 
as a surrogate for a school's economic composition. Since entry into the 
free and reduced-price lunch program is deteirmined by family income and sr'.ze, 
the percent of students who participate in this program provides an indicator 
of the percentage of poor students at each school. This variable alone 
accounts for 50 to 80 percent of the variation in average scores in each 
grade lovel, even though the variable represents the average participation 
of an entire school rather than an* individual grade. 

To produce signals of relative performances averages by :^rade on one 
reading and one arithmetic subtest of the achievement test battery for a given 
year were plotted against the amount of participation in the free and reduced- 
price lunch program at each school. The result of this effort is one scat- 
tergram for each grade and eacn subtest (7 grades x 2 subtests = 14 scatter- 
grams for each school) for each year. Figure II-l illustrates the relation- 
ship which exists between these two variables. The horizontal axis of each 
scattergram represents the percent of students at any school participating 

2. It is possible to' produce signals using the 'results of any achieve- 
ment subtest. 



ERIC 



II-9 



Figure II-l 

Relationship Betweei Achievement and Subsidized Lunch Participation 
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Reading Grade 5 



Low Medium High 

FREE AND RED13CED--PRICE LUNCH PARTICIPATION 

in the subsidized lunch program. The right end of the scautergram is 
labelled "High" participation, indicating those schools which enroll more 
students from poorer economic backgrounds. Each mark (+) on the scattergram 
in Figure II-l represents the grade equivalent average for all students who 
took the reading subtest in the 5th grade at a particular school, plotted 
against free and reduced-price lunch participation for that ertire school. 

Atlanta administers achievement tests in the seventh month (April) of 
each school year. The national norm for 5th graders who took the test at 
that time is defined by the test manufacturers as 5.7. It is apparent when 
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looking at the scattergram that most 5th grade average scores in Atlanta 

elementary schools fall below the n^^cional nurm. Jhe signals developed in 

this project do not compare individual school averages with the national 

norm; rather, the signals are derived from comparisons of schools in Atlanta 

which have student populations of similar economic composition. 

The process of making relative comparisons has been made easier by 

3 

fitting a curve labelled C through the data points. Figure II-2 below 
shows the resulting curve for 5th grade reading in one year. Any point on 
the curve may be thought of as the norm for schools which fall above or below 
that point. Figure II-2 also includes four other curves, two above (A and B) 
and two below (L and E) that basic fitted curve. These fo^r curves represent 
boundaries beyond which the average performance of students in that grade is 
considerably abcve or below the level of average performance that might be 
expected of students in similar schools. Boundaries have been designed so 
that only clear cases of extremely high or low relative performance are sig- 
naled. Thus, only about 10-15 percent of the grades in 3 11 schools are sig- 
naled in any one year. 

Five categories of signals result from the approach illustrated in 
Figure II-2. When the level of relative performance falls within the shaded 
range shov-m on Figure II-2, the grade is not considered an extrem<^ case and 
receives no signal. A grade in which the level of performance falls between 
the curves labelled A and B is signaled with a blue Sjemi-circle. When the 
level of performance falls on or above curve A, then the grade is signaled 
with a full blue circle. When the performance of a particular grade is rel- 
atively high, but the absolute level of performance is below the national 

norm, then the symbol 69 is placed in the appropriate full blue or half-blue 

3. The curve used here is a least squares fit. Documentation for the 
production of signals may be found in the Appendix. 
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Derivation of Signals 
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4 

signal. ' When the level of perf orrriance falls between curves D and E, then 

the grade is signaled with a red semi-circle. Wien the level of performance 

falls on or below curve E, then the grade is signaled with a full red circle. 
Since only performance among similar schools is being compared, two 5th 

grades with the same absolute grade equivalent average may receive different 
signals. A 5th grade which has a grade equivalerit average of 4.9 in a school 
with many poor students might be considerably above the point of the perform- 
ance curve for schools similar in terms of economic level; it would therefore 
receive a half-blue signal. However, a 5th grade which has the same grade • 
equivalent average of 4.9 in a school with very few poor students might be 

4. This distinction between relative and absolute performance was made 
at the suggestion of former Superintendent John W. Letson. 
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considerably below the performance curve for similar schools and it would 
receive a half-ied signal. 

The signals for each elementary, primary and middle school in Atlanta 
were organized into compact displays similar to the ones which appear on the 
following page* Figure 11-3 illvstrates the initial format for displaying 
signals for an individual school which was distributed to Atlanta in November 
1972. The format in Figure II-4 presents an additional year of information 
for the same school and was distributed to Atlanta officials in August 1973, 
at the close of this project. Each set of school signals was accompanied by 
a similar display of the mean achievement score for each grade, subtest and 
year. 

Pxesentatiofi of data in this format enables the user to see quickly the 
relative status of performance at a school. Several important facts about 
perfomance in Atlanta's schools emerge from this approach to the use of 

achievement test data. First, most grades in a school are not signaled 

I 

because the signaling system locates only extremes of performance, conserva- 
lively defined. Second, in three years of signals for approximately 130 
schools per year, there is not a single school in which every grade is sig- 
naled in both reading and math. Moreover, the pattern of relative performance 
within any school usually differs from one subtest to the other; the relative 
performance of students within the same grade on different subtests is also 
markedly dissimilar. 

Reading down a single column of one grid pictures the relative perform- 
ance of all grades in the school in reading or arithmetic at one point in 
time. Reading across a row of one grid pictures the relative performance of 
different groups of students in the same grade over time. For a school in 
• which student mobility is low, reading dpwn a diagonal compares the relative 
performance of the same group of students over time. 
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Another display of signals was prepared for use primarily by staff at 
the Area Office level. The reading and arithmetic signals for every school 
in an Area for one year were arrayed on a sinf^le sheet of paper. Staff in 
the Area Office and elsewhere throughout the central administration of the 
school system received a set of Area sheets for each year since 1971. This 
method of displaying signals enables an administrator to see at a glance 
the relative performance of every grade in all schools (as many as 30) in 
an Area. 

The next section of this chapter describes in more detail some of the 
properties of signals. 

D. PROPERTIES OF SIGNALS 

' Now that the technique for the production of signals has been described, 
some questions about the properties of signals can be examined. Signals have 
been produced for three years and will be used to examine the following 
questions: 

• What proportion of grades are signaled as cases of 
extreme relative performance in each of the three 
years? 

• Are the extremes of performance which are signaled 
of significant magnitude to be considered extreme 
performance? 

• How does the assignment of sigrals from year to year 
compare with that which would be expected from chance 
alone? 

• Was the observed pattern of signals produced in 1973 
diffe'^ent from that expec':ed .on the basis of signals 
from previous years? 

As the material which follows will show, signals do indeed indicate cases of 

extreme performance, even when measurement error is taken into account. The 
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pattern of signals is measurably different from the pattern vhich would have 
been obtained through cRance alone. Morebvar, for Grades 3 through 7, the. 
pattern of signal change from year to year did not alter markedly after the 
signals were introduced into the Atlanta system. 



1. THE ANALYSIS FEAI-EWORK 

In order to consider the questions posed above, an analysis framework 
was adopted which produced a consistent set of daca from year to year. A 
total of 115 schools have been included in the analysis set. The set 
includes only elementary schools which remained open (did not open or close) 
over the three-year period in question. The basic element selected for 
analysis is the *grade level^ (i.e.. Grades 1, 2, 3, 4, 5, 6 and 7) at each 
elementary school. The analysis framework thus consists of 115 schools, 
each of which in general has seven grades for a total of 115 x 7 = 805 cells. ^ 
Since a few schools do not contain all seven grades, tabulations will gen-- 
erally be based on less than the maximum number of cells. 

Figure II-5 illustrates the analysis framework. The shaded row (School 
004) indicates the seven grade levels withir. a single school. The shaded 
square is the 4th grade at School 302. Each column represents a single grade 



5 . rLnchine p;r^diii^ of the test results from Grades 1 and 2 for 1973 
were subcontracted by Atlanta to the test publishers. Due to some confusion 
on format and marking, several thousand test results were not graded, ren- 
dering these two grades unusable for analysis in 1973. V/hile there is some 
chance of recovering these data for further use, they were not available for 
use in this report. Because of this, all data have been converted for this 
.chapter to consider only Grades 3, 4, 5, 6 and 7. Data for Grades 1 and 2 
from 1970-71 and 1971-72 have already been processed at the Institute and 
are available. 

6 . A cell is a particular grade level at a particular school. Several 
schools in the analysis set no longer have 6th or 7th grades; they have been 
transferred to middle schools. Occasionally, test data are missing for a 
cell, even though the grade did exist. When considering only Grades 3, 4, 
5, 6 and 7, the maximum number of cells in the analysis set is 575. 
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level for the entire analysis set. Each cell in the .:igure represents a 
grade level at a particular school* This figure, indicates the finest level 
of detail considered,^ the level to which a particular signal is assigned for 
high (blue or half blue) , average (neutral) , or low (red or half red) 
relative performance. For each cell, signals are created for the reading 
and the arithmetic problem solving subtests in the spring of each year* 



Figure II-5 
The Analysis Framework 
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2. THE PROPORTION OF GRADES SIGNALED 

This section addresses the question of what proportion of grades are 
signaled as cases of extreme relative performance in each of the three years. 
The data presented in this section are based only on Grades 3-7, because of 
the scoring problems encountered in April 1973 for Grades 1 and 2. In each 
year since April 1971, a signal of relative performance in reading and in 
arithmetic problem solving has been produced for every grade 3-7 in the 
115 schools in the analysis set. 

A primary purpose of the signals was to help school officials locate 
cases of extreme relative performance. The limits originally used in 
developing the signals were designed to flag only about 15 percent of the 
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cells as extreme. It was felt that school officials could only respond to 

a minority of cases and that attention should be focused on the grades which 

were known to show large variation from some "norm." 

The signaling boundaries were selected in order to signal the grades at 

schools lying furthest from the curve fitted through the mean scores o^ all 

schools for each grade and subtest. Percentages were used because inspection 

had shown that there was generally more dispersion of scores around the curve 

for non-poor schools (where the absolute score was larger) and generally 

lower dispersion around the curve for poorer schools (where the absolute 

score was smaller). Thus, the use of a percentage boundary tended to 

equalize the numbers signaled from various economic categories. 

Table II~1 shows how signals were distributed in each of ^the three years, 

for reading and arithmetic problem solving • ''Blue" refers to' either full or 

half-'blue signals; "red" refers to either full or half-red signals. "Neutral" 

means the level of performance in the grade was not considered extreme and 

received a neutral signal. 

TABLE II-l 
DISTRIBUTION OF SIGNALS FOR GRADES 3-7 





READING 


ARITHMETIC PROBLEM SOLVING 


YEAR 


Blue 
# % 


Neutral 
# % 


Red 
// % 


Total 
# 


Blue 
// % 


Neutral 
// % 


Red 
# % 


Total 

// 


1971 


46 8.2 


480 85.7 


34 6.1 


560 


24 4.3 


518 92.5 


18 3.2 


560 


1972 


57 10.4 


454 83.1 


35 6.4 


546 


33 6.1 


500 '91.7 


12 2.2 


545 


1973 


.56 10.4 


453 84.3 


28 5.7 


537 


36 6.7 


493 91.8 


8 1.5 


537 



Of the cells signaled blue in reading, seven received blue sig- 
nals in each of the three years. Five additional grade cells received blue 
signals in reading in both 1971, and 1972, but not in 1973; seven additional 
cells received blue signals in 1972 and 1973, but not in 1971. No grade 
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cell in Atlanta received a red signal in reading three years in a row. Six 
grade cells received red signals in 1971 and 1972; seven other grade cells 
were red in 1972 and 1973 in reading. 

Table II--2 below provides data on the amount of overlap occuring between 
reading and arithmetic signals. Each grade receives two signals: one in 
reading and one in arithmetic. There is a smaller number of grades sig- 
naled in arithmetic than in reading. 

TABLE II-2 

OVERLAP OF READING AND ARITHMETIC SIGNALS 
FOR GRADES 3-7 



NO. OF GRADES WITH BLUE SIGNALS IN: 


NO. OF GRADES WITH RED SIGNALS INT- 


YEAR 


Reading 


Arithmetic 


Reading and 
Arithmetic 


Reading 


Arithmetic 


Reading and 
Arithmetic 


1971 


46 


24 


20 


34 


18 


11 


1972 


57 


33 


19 


35 


12 


7 


1973 


56 


36 


24 


28 


8 


5 



Data in the table show that a grade which receives a^blue or a red 
signal in arithmetic is likely to receive the same kind of signal in reading 
about two-thirds of the time. There are, of course, many grades signaled 
(red or blue) for reading that do not receive the same signal for arithmetic. 
Further examination of actual scores would be necessary to reveal the extent 
of differences in relative performance between reading and arithmetic. 



3. HOW EXTREME IS THE PERFORMANCE SIGNALE D? 

One way to deal with the question of how extreme is the performance 
which is signaled (either red or blue) is to consider how far away from the 
basic curve each red or blue signal lies. An examination of some actual 
signaled cells will be used to illustrate what has been learned about how 
far signaled cells lie from the basic fitted curve. 
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In the process of fitting a curve through the achievement data for each 
grade, each year and each subtest, the distribution of the distance that the 
average scores fell from each fitted curve was collected. The standard error 
(cr) value for this dispersion of score's was obtained in each case. For exampl 
the distance of the average score of all children in each red signaled grade 
in reading in 1972 from the fitted curve or "norm" for that grade, subtest, 
and year was measured. The variation of the average of a grade from the 
curve (expressed in grade equivalent units) was divided by the standard 
error (cr) value for all points about that curve to determine how far away 
from the curve the extreme cases tended to fall. The results showed that, 
of 35 red reading signals for 1972, 10 were more than 2cr below the curve. 
An additional 24 were between and 2cr below the curve- The remaining 
signal was 0.95cr below the curve; this nearest case came from a lower grade 
(3rd) at a relatively poor school (100 percent free and reduced-price lunch 
par'ticipation) . These results appear to hold for the other sets of signals. 
For the signals based on arithmetic problem solving,- the limits used to 
produce the signals turned oat to be even more conservative. In arithmetic, 
fewer cases were signaled, and those that did receive either a blue or a 
red signal were located even further from the curve than reading signals. 

In absolute grade equivalent terms, the standard error of the distribu- 
tion of scores around the curves in 1972 ranged from 0.69 grade equivalent 
in 7th grade reading to 0.28 grade equivalent in 3rd grade problem solving. 
Percent boundaries were used in signaling because dispersion of the averages 
about the curve is not uniformly distributed and tends to decrease as percent 
free and reduced-price lunch participation increases. As an estimate of an 
average distance used in signaling, the 15 percent signaling boundaries are 
0.9 grade equivalent units from a curve at a grade equivalent score of 6.0 
and 0.45 grade equivalent units at a grade equivalent score of 3^0. 
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The next question about the signaled cells is how accurately were their 
average scores measured • That is, even though the boundaries were sufficiently 
conservative, what effect did measurement error have on the signal assigned to 
a particular cell? The standard error of measurement for a single student on 
the achievement tests used by Atlanta was approximately 0,4 grade equivalent 
in 3rd grade and 0.6 grade equivalent In 7tli grade The average alze of a 

grade ranges from 50-60 children. This suggests that the average on the gradp 
is measured within 0.06 (or 0.4/^ 50) to perhaps 0.08 (or 50) grade 

equivalent at one standard error. It is doubtful that this small an error 
occurs in practice; however, if an error of this range did occur, it would 
not disturb the estimate that most of the cases signaled are extreme. It 
would simply mean that on remeasurement csome alternative extreme cases might 
be signaled near the boundaries. 

The discussion of measurement error will become more important when 
transitions of the signals from year-to-year are considered. Up to this 
point, the discussion has considered only whether extreme cases are being 
signaled. When only signals f'^r one year are considered, it is less impor- 
tant that some legitimately extreme performers are not signaled as long as 
those signaled are, in fact extreme. When signal changes from year-to-year 
are considered, it is of interest how many cells have average values near 
enough the boundary between extreme performance (red or blue) and average 
performance (neutral) that they could switch signals simply through measure- 
ment error. 

In the case of signal transitions from year to year, a quick test has 

been made by taking a census (through visual inspection) of those averages 

which fell near the inner boundary lines in 1972 reading. To estimate both 

7. See, for instance, Durast et. al . ,^ T eacher's Handbook^ Metropolitan 
Achievement Tests , Harcourt, Brace and Jovanovich, New York, 1971. 
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a best and worst case, a count was made of the total number of cases (Grades 
3, 4, 5, 6 and 7) fallxng within +0.06 grade equivalent and +0.15 grade 
equivalent of the inner boundary lines. The results are given in Table 
II-3 below. 

TABLE II-3 

TABULATION OF CASES NEAR THE BOUNDARIES IN 



1972 READING FOR 


GRADES 3 - 


7 






BOUNDARY 


+0.06 


+0.15 


-0.06 


-0.15 


Rlue/Neutral Boundary 


10 


21 


10 


26 


Red/Neutral Boundary 


11 


44 


8 


17 



Consider first the red boundary. There are eight red signaled grades 
within 0«06 below the boundary and 17 red signaled grades (including the 
previous 8) within 0.15 below the boundary, out of a total of 35 red signaled 
cells. If we postulated as a worst case that the 17 signals represented the 
Icr xrange of measured value for a larger set of signals that all lay on the 
boundary, then the total set would contain approximately 51 signals. [This 
assumes a normal distribution in which a single lO" contains -^17 signals 
and the total set, therefore, contains ^51 signals.] Approximately one- 
half of these, or about 26, V70uld be expected to fall in the red signal 
range. One-half of the 26 (again on average) might be expected to change 
from red to neutral simply due to measurement error in the next cycle of 
measurements. That would mean that 13 of the red signals become neutral 
simply due to remeasurement . While they v^nuld still be relatively extreme, 
they would not be signaled red in the remeasurement. 

Clearly, both the error range and assumptions used have been made 
extreme in this case to show how many of the 35 red signaled cells might 
be signaled neutral during a remeasurement. Even. in this worst case, the 
probability is 13/35 (or 0.37) that a red signal would become neutral. 
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If the Indicated range of 0.06 had been used as icr, and the same procedure 
followed, the number of potential switches would have been 6 of 35 or 17 
percent. Using the same assumptions for blue, 7.5 (13 percent) to 16 (28 
percent) of the 57 blue signals would become neutral in a remeasurement . 
These calculations will be referred to again when transition probabilities 
are discussed later in the chapter. 

4. IS THE ASSIGNMENT 07 SIGNALS DUE TO CHANCE? 

Since changes are always being made within a school system, it is 
extremely difficult to predict how much stability can be expected in the 
signals from year to year. School officials could provide no yardstick by 
which to predict the effects on relative performance of changes in teachers, 
shifts in student populations, -^nd the continuous modification of the 
instructional program. If all such changes were beneficial, then increasing 
stability in blue signals and less stability in red signals (within the 
limits of measurement error) might be predicted. The analyses conducted to 
date indicate that most actions by school officials produce diverse effects. 

It is important for the reader to realize that this is quite different 
from saying that no action by school officials makes a difference. It may 
be that present constraints and lack of systematic purposefulness create 
effects which cancel out. Consider the hypothetical and oversimplified 
example of a teacher-caused red signal. Suppose that the poor teacher is 
replaced with an excellent one and, in the next school year, the signal 
for that grade becomes neutral or blue. The poor teacher is not likely 
to have left the school system. Instead, the poor teacher is likely to have 
been reassigned to another school. This transaction could result in the 
creation of a new red signal (all other things being equal). Without better 
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attribution or more detailed analysis, the lack of systematic improvement 
throughout the school system would lead one to conclude (as our early exam- 
inations indicate) that teacher transfer' does not lead to improvement. 
The point is that this lack of effect in the statistics does not necessaril 
mean that the teachers make no difference. It can as easily mean that the 
control mechanism works in such a way that no net change is produced. 

The allocation of signals from one year to the next will now be 
examined to determine if they could have been assigned by chance alone. Mo£ 
statistical tests address the question of whether the assignment of signals 
to cells is significantly different from a chance distribution. The data 
which follow address the following hypothesis: 

• The signals in 1972 were assigned to cells at random and were 
not conditioned upon the signal given to that cell in 1971. 

Table II-4 belov; contains a matrix which shows the distribution of reading 

signals for 1972 based on the reading signal received in 1971. 

TABLE II-4 

ACTUAL SIGNAL OCCURRENCES FOR 
GRADES 1-7 IN READING, 1971 AND 1972 

1972 



signal; 


771 


Blue 


Half 
Blue 


Neuxral 


Half 
Red 


Red 




32 


54 


624 


53 


8 


Blue 


15 


3 


3 


9 


0 


0 . 


Half Blue 


62 


8 


9 


45 


0 


0 


Neutral 


633 


19 


40 


527 


42 


5 


Half Red 


42 


2 


2 


29 


8 


1 


Red 


19 


0 


0 


14 


3 


2 



This matrix was divided into zones for initial testing of the hypothesis r.s 
follows : 
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TABLE II-5 
ZONE DEFINITION 

1972 



8 



1971 



SIGNAL'. 


771 


Blue 


Half 
Blue 


Neutral 


Half 
Eed 


Red 


32 


54 


624 


53 


8 


Blue 

Half Blue 


15 
62 


A 


B 


C 


Neutral 


633 




E 




Half Red 
Red 


kl 
19 


D 


F 


G 



KEY : A - Blue both years 

B - Blue in '71, Neutral in '72 
C - Blue iu '71, Red in '72 
D - Red in '71, Blue in '72 



E - "Neutral both years 

F - Red in '71, Neutral in *72 

G - Red both years 



For the hypothesis stated above to be correct, the signals in 1972 snould 

be distributed proportionately among cells and any particular grouping of 

cells (such as the zones above) should not affect this. 

Consider Zone A, which is composed of cells signaled either full or 

half -blue both years. There were 77 such cells in 1971. If the signals 

wiere randomly and independently assigned in 1972, then the probability of 

32 + 54 

any cell receiving a blue signal is — — = p = 0.112, If the binomial 
distribution is used as an approximation, then the expected number of blue 
cells in Zone A is = np = 77 (0.112) = 8.6 and the standard deviation (cr) is 
o* = %/^p(l-p) =V77(0.112) (0.888) = 2.8. That is, if the hypothesis were ' 
true (and two possibilities in 1972 are considered; blue or not blvte -For t^e 
cells that were blue ' \ 1971) then we would expert to find in Zone A about 
8.6 blue signals in 1972 with a standard deviation of 2.8. In fact, there are 
23 blue cells in 1972 — or about five standard deviations from the mean. 



8 . The zones "neutral to blue" and "neutral to red" have not been 
carried along in this example. Therp are many different causes of these 
transitions and they generally occur at or near a chance rate. 
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The ■ .suits of similar analyses for the remaining zones are presented in 
Table 11-6 (for ^-eading) and Table II-7 (for arithmetic). The probabilities 
for each case are not shown. Instead the results are presented in units of 
standard deviations from the mean. The upper portion of each table displays 
the predicted mean (/x), standard deviation (a) and the observed mean (/x) for 
each zone. The lower portion of each table gives the value of Z (where Z = 

^) for each zone. Where the normal is a* good approximation, a unit normal 
value of Z that is a little larger than 1.6 would be representative of a one- 
tailed test at the .05 level of significance;, for a two-tailed test, the .05 
levsl would be 1.96. The diagonal running from upper left to lower right 
represents the tendency for the cell representing the same grade to have th^^ 
same signal in a succeeding year. The other diagonal represents the tendency 
of the signals to shift to the other extreme (e.g., red to blue, blue to red). 

On the basis of the value summarized in Tables II-6 and II-7, the 
hypothesis that signals in 1972 were assigned at random can be rejected in 
many of the zones witnuut adjusting for other descriptive variables such as 
student mobility or staff changes. In zones A and G, the high Z values 
indicate that assignments of signals in 1972 were not random but were depen- 
dent upon the signal received in the preceding year. The positive sign c:i Z 
indicates that more signals were assigned in 1972 to these cells (i.e., more 
remained high or low) than would have been expected by chance. Negative 
signs indicate that fewer occurrences existed than would have been dictated 
by chance. ■ I^en examining negative Z*s, the reader will also r.sed to examine 
/X and cr , since in some cases a nearby boundary (0 or no cases) leads to a 
value for Z that is not as large as might have been expected. 
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TABLE I 1-6 

VALUES FOR THE PREDICTED mm, THE OBSERVED MEAN, 
THE STANDARD DEVIATION, AIJD Z-VALUES, IN READING, 
1971 AND 1972 



a. t^,o-,fi: 

1972 



SIGNAL 


HALF 

BLUE BLUE 


NEUTRAL 


HALF 

RED RED 


TOTAL 


B 


M - 8.6 
cr = 2.8 
^^ = 23 


At= 62 
cr=: 3.5 
54 


M = 6.1 
0- = 2.4 
1^ = 0 


77 


HB 


N 




M-512.0 
cr- 9.9 
A=327.0 




633 


HR 


M - 6.8 
cr - 2.5 


M= 49 
cr= 3.1 
a= 43 


M 4.8 
<r « 2.1 
U 14 • 


61 


R 


TOTAL 


86 


624 


61 


771 



b. Z-Values: 



1972 



1971 



SIGNAL 


BLUE 


NFUTKAL 


HALF 
RED 


TOTAL 


B 


^ = 5.1 


\ 

2 « -2.4 

I 


Z- = -2.5 


77 


HB, 


N 




Z = 1.5 




633 


HR 


2 = -1.1 


E = -1.9 


H «= 4.4 


61 


R 


TOTAL 


86 


624 


61 


771 



i 
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t;:I;li?: li:-: 

li'/i AND 1972 . 



a. /X , cr. A '■ 

1972 





MW.h f 


HALF 


TOTAL 


B 


M « A, 9 
cr - 2.1 

f 27 


M« 50 
o-- 2.7 

/2= 29 


M •= 3.2 
0- » 1.7 

^0 1. 2 


58- 


KB 






567 
cr« 9*0 
A« 600 




660 


HR 


<r = 2.0 
0 


« 

or» 2.5 
a« 32 


/t « 2.9 
0- « 1.7 
^ « 20 


52 


R 


TOTAL 


66 


661 


^3 


770 



b. Z-Values: 



1972 



SIGNAL 


^^^^ 1 BLUE - 


KEUTRAL 


HALF 
RED 


TOTAL 


' B * 


& « 10.5 


Z « -7.7 


55- - -0.7 


58 


HB 






Z « 3.7 




660 




2 B -2.2 


Z « -5.2 


.2 « lO.p , 


52 


R 


TOTAL 


66 


■ -Cor ■ 


/ " 43 . 


770 
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Many cells do change signals from year to yt»ar. This can come fron a 
combination of effects from teacher transfer, pupil transfer, error of 
measurement, change of teaching techniques or curriculum, etc. For instance, 
it was observed earlier that changes of ' perhaps one"-fourth of the signals might 
come about through measurement error. In addition, a rather random method 
of managing changes in^ the educational program and faculty may produce 
apparently random results when gross statit>tical techniques are used. Even 
with these present, this analysis indicates that there is a nonrandom 
mechanism at work selecting signals- That is, knowing the signal of a grade 
in 1971 definitely gives probabilistic information about the signal received 
by the grade in 1972. Therefore, the hypothesis of chance assignment is 
rejected. 

5. DID TRANSITION BEHAVIOR CHMGE 
THIS YEAR ? 

As stated before, changes in the level of performance from one ye^r to 
the next are tc be expected in a school system. Students change, as does the 
curriculum, teaching staff, and/or materials used in a grade at a school. 
Moreover, the school system may already be taking steps to alleviate or 
replicate conditions which give rise to signals. This section addresses 
the patterns of allocation of signals of relative performance for 1971, 1972, 
and 1973 and how the transitions from year to year differ. Since data also 
were available characterizing the changes that occurred between 1971 and 1972, 
it was possible to compare the transitions in performance signals made at 
grade levels at individual schools from spring 1971 to spring 1972 with the 
transitions made from spring 1972 to spring 1973 (the period during which 
signals were introduced) . 
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Table II-8 presents data describing the transitions of signals in 
reading for Grades 3, 4, 5, 6 and 7 of the 115-school analysis set. The table 
includes all grades that were present and signaled in both years (544 grades 
or-cells). To understand Table II-8, think' of the signals as being distrib- 
uted to cells of grade levels at schools by the signaling technique for 1971. 
Table II-8 shows that in 197l~of the cells signaled in both years (544) — 44 
cells received blue signals (full blue or half blue) , 467 cells received 
neutral performance signals (no color), and 33 cells received red signals 
(full or half red). Of those same 544 cells in 1972, 57 cells were signaled 
blue, 452 cells were ntiutral, and 35 cells were signaled red. 



. TABLE II-8 

ASSIGNMENT OF READING SIGNALS, 1971 TO 1972 

1972 



SIGNAL: 


Blue 


Neutral 


Red 


Total 


Blue 


12 


32 


0 


, 44 


_Neutral _ 


_ 44 


_ _394 _ . 


- J9_ 


__467 _ 


Red 


1 


26 


6 


33 


Total 


57 


452 


35 


■ 544 



Observe that of the 44 cells that were blue in 1971 (top row of data), 
12 remained blue in 1972, 32 received neutral signals, and no cell became 
red in 1972. Table II-9 displays the data converted into a probability of 
transition matrix for reading, from 1971 to 1972- The matrixshows the 
probability of making any of the possible transitions from one signal cate- 
gory to another, based on the data from 1971 and 1972. For instance, the 
probability that a blue in 1971 would remain biue in 1972 was 0.27. 
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TABLE II-9 

.TRANSITION PROBASaiTIES FOR 1971- 
1972 READING (GRADES 3-7) 
1972 



1971 



SIGNAL: 


Blue 


Neutral 


Red 


Blue 

Neutral 

Red 


_ 'JJ. _ 
_.09 __ 

.03 


_J1 _ 
_ . 8_5 

.79 


q _ 

.06 _ 

.18 



Consider this matrix as providing the basis for estimating the tran- 
sitions vhich might be expected between signals in 1972 and, those which 
occurred in 1973. For instance, consider the set containing the 57 cells 
that were blue in 1972. If these 1971-72 transition probabilities were to 
apply in 19712-73, then the following distribution of signals would be 
expected for those 57 cells: 

57 X (0.27) = 15.4 would remain blue, 

57 X (0.73) = 41.6 would become neutral, and 

57 X ( 0 ) = 0.0 would become red. 

The actual numbers of signals for 1973 were 14 blue, 42 neutral, and one red. 

9 

These numbers differ by less than lo" from those expected and demonstrate 

that the 1972-73 transitions of blue signals are predictable using the 

1971-72 transition probabilities for blue signals. The implication of this 

finding is that there was no sharp change in the transition behavior of blue 

2 

signaled cells from 1972 to 1973. With these actual numbers, a simple X 

test with one degree of freedom using the actual number yields the following 

value: o 7 

X2 ^ (14-15.4)^ (^3-41.6)^ ^ 
15.4 ^ 41.6 

~9. Where (X is defined as ^np (1-p) = V57(.27) (.73)'^3.3. 
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indicating again that the observed data in 1973 are distributed about as 
would have been predicted from the 1971-72 transition probabilities. 

This kind of examination was extended to cover the transition in signals 
for reading between April 1972 and April 1973. Those cells from the 1972 
signal set present in the 1973 signal set must be used in developing the 
expected transitions from 1972 to 1973, A total of 534 cells^^ were present 
in both April 1972 and in April 1973. The division of the 534 cells (present 
in 1972 and 1973) into signal categories in 1972. was as follows: 57 were 

blue in 1972, 443 were neutral in 1972 and 34 were red in 1972. The total of 
534 cells is slightly lower than the total in the 1971-72 trans itioxi set be- 
cause" some 6th and 7th grades were transferred to middle schools in 1973 and, 
hence, were excluded from the analysis set. Applying the transition probability 
matrix to these data, the transition table for 1972/1973 would be expected 
to resemble Table 11-10. [The projections have been rounded to whole numbers 
for display here. ] 

TABLE 11-10 

PREDICTED TRANSITIONS OF READING SIGNALS 
IN 1972-73 BASED UPON 1971-72 DATA 
1973 



SIGNAL: 


Blue 


Neutral 


Red 


Total 


Blue 


15 


42 


0 


_57_ 


Neutral 


40 


376 


27 


443 


Red 


_ 1 _ 


_ 27 _ 


_ 6_ 


_34_ 


Total 


56 


445 


33 


534 



Table n-H then compares the observed transition counts with those pre- 
dicted for 1972-73 • The largest variation is in the case of neutral to red ' 

[there are fewer reds observed than were predicted] ; this is not 

10. , Data for Grades 1 and 2 were not yet available for 1973 and there- 
fore the number of cells is reduced to 534, 
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statistically significant. For the school system as a whole, therefore, 
there is no indication of a large change in transition behavior in reading 
between the two periods 1971-72 and 1972-73. 

TABLE 11-11 ■ 

A COMPARISON OF PREDICTED /OB SERVED 
TRANSITIONS BETWEEN 1972 AND 1973 

1973 



1972 



SIGNAL: 


Blue 


Neutral 


Red 


Blue 


'l4 


^42 




Neutral 




376, 
'383 


'20 


Red. 




'26 





A similar -examination can be made for arithmetic problem solving signal 
transitions. Here, lower numbers of red and blue signals make the results a 
little harder to^ rely on statistically. • - 

Table 11-12 presents outcomes for arithmetic similar to that pre- 
viously reviewed for reading. The probability of transition for the 1971-72 
case has been used to predict outcomes in 1973. The predicted ^distribution 
(rounded to whole numbers) is shown In Table 11-12. 

TABLE 11-12 

PREDICTED TRANSITIONS OF ARITHMETIC SIGNALS 
1972-1973 BASED UPON 1971-1972 DATA 
1973 





SIGNAL: 


Blue 


Neutral 


Red 




Total 


1972 


Blue 

Neutral 

Red 


13 

_ 24_ 

_ 0 


19 

_^59_ _ 
9 


1 



3 




33 
488 
12 




Total 


37 


487 


9 




533 
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Table 11-13 displays the comparison of the predicted transitions for 
arithmetic signals to the observed signals. In this case, there Is some- 
what more variation than In reading. Both blue-to-blue and red-to-red 

show smaller populations than predicted. The difference In the blue dlstri- 

2 

butlon would be statistically significant =3.25 with 1 degree of free- 
dom) at the 0.1 level, but not at the 0.05 level of significance. That is, 
there are fewer blues remaining blue than would have been predicted from 
1971-72 data. 

TABLE 11-13 

A COMPARISON OF PREDICTED/OBSERVED 
TRANSITIONS BETWEEN 1972 AND 1973 
1973 



SIGNAL: 


Blue 


Neutral 


Red 


Blue 




'25 


\ 


Neutral 


"/ay 


459. 
'454 




Red 


\ 




\ 



There are also fewer reds remaining red than nilght have been predicted, 
although not statistically significant (the tables contain rounded data and 
the actual expected values are red-to-neutral = 9. A and red-to-red = 2.6, 
where a binomial approximation of cr =\/npq = ^.12 (•22) (.78) -^1.4) . Since, 
the reduction is also somewhat symmetrical, one might also wish to examine 
the residual error at each point, if any significant behavior conditioned on 
signals had been found that might affect problem-solving signals in this way.. 
Since no signal-conditioned actions were found that were likely to have 
caused- this change, the behavior may be a function of the particular 
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signaling technique in use on these data, or a function of test differences, 
or a reflection of the myriad of changes taking place in a fairly unsystematic 
way within the school system* 

6. CONCLUSION 

These above illustrations display some of the properties and 
behavior of the operational signaling system in producing signals of cases 
of extreme relative performance. The signal curves illustrate but do not 
directly address some additional important policy questions raised by the 
distributions, of achievement scores within the Atlanta School System. For 
example, the shape of the fitted curve shows that poorer schools tend to have 
increasingly lower achievement scores. That this is generally true in school 
systems has been documented by extensive national research, l^hether lower 
scores in poorer schools are acceptable and whether methods are available to 
alter this condition in a local school system are policy questions of some 
magnitude. Signals provide no answer to these policy questions. Rather, the 
signaling technique merely points out that this condition seems to recur year 
after year. 

What the signaling technique does adjust for is the economic level 6f 
the students at each school and, in essence, it creates a new "norm" (the 
fitted curve) for all schools based upon the economic level of the students 
at each school. Once this is done, it is seen that some grades at some 
schools still exhibit high or low performance relative to other schools with 
similar student populations.. Signaling isolates these cases of high and low 

11. There is some indication that the mathematical problem solving 
results may be distributed somewhat differently on. the Metropolitan Achieve- 
ment Tests than on the Iowa tests-. The Department of Health, Education and 
Welfare had not released the results of its "anchor test" , study of test 
comparisons in time for their use in this report. Later work may be able . 
to consider this. ' * 
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relative performance so that they are brought to the attention of the school 
system. That Is, signaling provides short-term Information: the locations 
In the school system where some extremes of relative performance are occur- 
ring after an adjustment has been made for the socio-economic characteristics 
of the student population. 

The next question for consideration Is how the relationships between 
signals and the decisions and actions of school officials were established. 
The next chapter explains the approach used In this study. 



CHAPTER III 

I 

THE EXPERIMENT: SETTING AND PROCEDURES 

The signals, described in the preceding chapter, were distributed to 
Atlanta school officials in tJovember 1972. The purpose of this chapter is 
to acquaint the reader with the Atlanta School System, to give the reader a 
perspective on the major events of the 1972-73 school year, and to describe 
the procedures used to monitor the activities of school officials in order to 
detect the impact of signal inf omation. 
■ 

A. CH ARACTERISTICS AND STRUCTURE 
OF THE ATLANTA SYSTEM 

When the schools opened in the fall of 1972, the Atlanta system had an 
enrollment of over 96,000 students, a teaching staff of 4,700 aud an annual 
General Fund budget (excluding capital funds and most federal funds) in 
excess of $97 millions. Enrollment » whicli peaked several years aso, had 
declined to 91 > 000 by the end of the school year. Black students accounted 

for over 75 percent of the total enrollment in 1972-73, and the proportion 

12 

continues to grow. 

Atlanta students typically attend element y scuool (Kindergarten through 
Grade 7) and high school (Grades 8-12). However, there has been movement 

^— •— — — — — ~ ^ 

12. In September 1973, enrollment had declined to 89,000 students (ap- 
proximately 80 percent black); the teaching staff had declined to 4,600 
positions. 
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toward a new organizational pattern which will divide students into elemen- 
tary (K-5) ; middle (Grades 6-8) and high- schools (Grades 9-12). During 
1972-73, there were 123 elementary schools, 5 middle schools and 25 high 
schools organized into five geographic areas. As total enrollment decreases 
the total number cf schools and the size of the teaching staff also decrease 

Figure III-l illustrates the organizational structure of the Atlanta 
system during the year' under study. Once every four years, voters in Atlant 
elect a Board of Education to set general policy for the school system, in 
close consultation with the Superintendent and his staff. The School Board 
has authority to raise revenues for the operation of the schools; no public 
approval is required for changes in the property tax rate, - 

The School* Board appoints the Superintendent and the other principal 
administrative officers of the system. The Superintendent of Schools pre- 
sides over the day-to-day operations of the schools, aided by a staff of 
six Assistant Superintendents, a Comptroller and five Area Superintendents. 
Unlike many school systems, Atlanta has had no Deputy Superintendent* Flex- 
ibility has been the key organizing principle of the school system. The 
major officials of the school system have operated in the absence of any 
written lefinitions of their responsibilities and functions. 

In general, the Assistant Superintendents and the Comptroller function 
as staff to the Superintendent. With one exception, each has been respon- 
sible for certain central activities associated vrith the operation of the 
schools • Each Assistant Superintendent provider services to schools and 
teachers which are usually filtered through the Area Office. Each Area 
Superinteni^ent serves as a line administrator, responsible for all aspects 
of the educational process occurring in the schools in his geographic area. 
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Ivhat follows is a capsule description of the responsibilities of each major 
administrator and/or division within the-Atlanta system as they existed in 
1972-73. 



1, Assist a nt Superintendent for Administrative Services . This division 
provides cenLralized services to the schools and to other parts of 
the central administrative structure. Major responsibilities include 
purchasing and distribution of expendable supplies, materials, fur- 
niture, textbooks and library materials; maintenance of school build- 
ings and grounds; operation of food programs; provision of school 
detectives to investigate crimes which occur on school property; 
maintenance of pupil records on attendarce, enrollment, and mobility. 

In Comptroller . The Comptroller is responsible for the formulation of 
Atlanta's annual General Fund budget as well as for the supervision 
of all payroll and accounting activities which accompany the adoption 
of the budget. Atlanta has adopted a program budget format for the 
preparation and accounting of its funds. The comp.uter facility is 
• part o'r the Comptroller's responsibilities. Pupil attendance, prop- 
erty inventories, payroll, textbook, library and audio-visual supply 
purchasing, some grade reporting, high school student scheduling and 
achievement test processing are all computerized in Atlanta, 

3, Assistant Superintendent for Instruction , The Instructional Division 
is the largest of the staff groups in the Atlanta structure. Al- 
though the division has about two dozen different components, its 
activities can be grouped into four broad categories: the develop- 
ment, maintenance and revision of curriculum; teacher training and 
staff development; the administration of special projects or p.:ograms 
(usually federally funded); and pupil services, including the admin- 
istration of activities for exceptional children, guidance, health, 
and social services, 

4, Assistant Superintendent for Personnel , This division is responsible 
for the recruitment, certification and placement of all school system 
personnel. Personnel directs the competitive process established 
for selecting individuals to fill any position above the level of 
the classroom teacher. Substitute teachers are recruited and cer- 
tified by Personnel (although the individual school principal is 
responsible for arranging to have a particular substitute cover an 
empty classroom), 

5, Assistant Superi nt endent for Pvesearch h Development , A primary 
function of the Research h Dev=lopment unit is the development of 
new federal projects for the school system, Sr.aff members prepare 
proposals and, once federal funding is secured, have responsibility 
for federally required evaluation of those projects. R&D coordinates 
all research projects underway in the school system. Staff members 
produce a newsletter, distributed throughout the school system, sum- 
marizing the results of selected educational research. 
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6. Assis tant Superintendent for School Plant Planning . This division 
is responsible for the acquisition of land foi future facilities, 
che planning and construction of all new facilities and major ad- 
ditions to existing facilities, major renovations, and for the 
equipping and furnishing of new buildings. Money for these activ- 
ities comes from a separate Capit.al. Funds budget. Most construction 
is done under contract to the school system, 

7. Area Superintendents , The fr've Area Superintendents serve as the 
direct link betx^7een the Superincendent , the rest of his staff and 
the schools. Each Area Superintendent has a staff of instructional 
personnel, called Resource Teachers, v/ho observe, assist and help 
train classroom teachers in the schools in that Area, The Area 
Superintendent figures prominently in every decision which affects 
the operation of a school: organization and staffing; recruitment 
and assignment of principals and teachers; selc :tion of schools as 
sites for special programs; equipping or remodeling schools; assign- 
ment of pupils; curriculum selection; budget preparations; community 
relations, etc. 



B. THE YEAR IN ATLANTA 



The introduction of information about relative performance was not the 
only event to disturb the i/orma.' routine of the school sycuem during the 
1972-73 school year. In fact, 1S72-73 witnessed major changes both in the 
organization of the school system and in the personnel responsible for its 
day-to-day operation. 

Throughout the entire school year, the Atlanta School System was involved 
in a school desegregation suit whose settlement effected the assignment of 
students and staff and resulted in a major upheaval in the administrative 
structure and personnel of the school system. Between October 7, 1972, wh^:in 
the Appellate Court first ordered the school system to come up with an ac- 
ceptable desegregation plan, and August 23, 1973, when that same Court finally 
upheld the plan (agreed upon by the plaintiff ..and the Atlanta School System 
in February 1973), for a period of one year, the Atlanta school community was 
in a 'stat^ of uncertainty about the scope and nature of any future arrange- 



ments. 
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Under the terms of the compromise plan finally accepted by the Court, 
no Atlanta school would have less than 20 percent black enrollment, although 
some all-black schools would remain* The required racial composition of 
Atlanta schools was to be achieved through a combination of techniques, in- 
cluding pairing of schools, voluntary student transfers, school closings and " 
some busing. In 27 schools, teachers would be. reassigned to alter the racial 
composition of the faculty • Over 2/3 of Atlanta's schools had been affected . 
by the compromise plan by the time school opened in September 1973. 

The compromise plan did not stop with the transfer of studerts and staff - 
A major feature of the compromise plan was the provision that at least 50 
percent of the top administrative positions, including that of Superintendent, 
were to be filled by blacks. To accomplish this, 16 new administrative posts 
were to be created. The search for a new Superintendent began in the middle 
of the school year and continued until July 1973, when Dr. Alonzo A. Grim 
was named to succeed Dr. John W. Letson. The 16 new positions remained vacant 
and their -responsibilities undefined, pending the arrival of the new Super- 
intendent. The process of selecting a new Superintendent, couplc^d with the 
potential expansion and reorganization of the top-level administrative staff, 
created an atmosphere of uncertainty among existing administrative personnel 
which resulted in a general reluctance to make decisions or to take actions. 
To a large extent, the school system marked time during much of the year. 

Quite independent of the desegregation plan, a number of other changes 
in key administrative personnel occurred during the year* For several 
years preceding 1972-73, none of the top 13 executive positions in the school 
system changed hands. Yet, within a 12-month period, beginning in August 1972 
five of the executive staff, including the Superintendent,, three Area Super- 
intendeSts arid the Assistant . Superintendent for Personnel, were replaced. 
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Even the School Board was not immune from change during 1972-7 3, In 
March 1973, the portions of the Atlanta city charter governing the method of 
election to the School Board were revised. Effective as of the October 1973 
election, the Atlanta School System will be governed by a nine, rather than 
a ten member School Board. The method of election of Board members was 
changed so that several of the then sitting Board members would have to step 
down or seek re-election from the same district. 

As mentioned earlier in this chapter, total student enrollment has 
declined in recent years. Moreover , there have been important shifts among 
the existing population. As a result, ::he distribution of schools and 
students among the five geographic Areas had become very unequal. In October 
1972, the Schocl Board adopted a plan to divide the system into six Areas, 
each serving approximately 15,500 students and 25 schools. Because of the 
uncertainty over the desegregation plan, that reorganization plan was aban- 
doned. Once the school closings and shifts of pupils necessitated by the 
desegregation plan were agreed upon, a new plan for organizing the school 
system was dravm up. The new plan, put into effect at the close of the 
1972-73 school year, equalizes the number of schools and students in each 
of five Areas. 

The administrative reorganization and the desegregation plan have had 
far-reaching effects on students and staff. Within the space of one school 
year, the following changes occurred! 12 schools were closed and 4 were 
opened; 2 high sc'iools were converted to m.iddle schools; 24 schools were 
affected by Area boundary changes. Alternative arrangements had to be made 
for students and teachers whose schools were closed. For those schools 
affected by the new Area boundaries, the new school year would mean a rew 
Area Superintendent, a new, set of Resource Teachers and, in general, becoming 
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accustomed to a different style of admiriTstration, For Area Office staffs 
the changes have meant -becoming f amiliar-with the program, staff and problems 
oi new schools. 

The disruption resulting '.rom the Court decision and the reorganization 
was compounded still more by a significant reduction in federal funds avail- 
able to Atlanta under Title IV-A of the Social Security Act, In December 
1972, Atlanta's share of Title IV-A funds for the next 12 months was reduced 
by about 80 percent (an amount equivalent to about 10 percent of the General 
Fund budget). The funds had been used to provide early childhood education 
programs, after-school programs for older children and a variety of supple- 
mentary services. . The school system had hired staff under one-year contracts 
at the opening v)f school, on the assumption that Title IV-A funds would con- 
tinue to be available at che 1972 level. The drast.'.c reduction in IV-A funds 
for 1973 forced school system officials to close down programs in mid-year 
and to find other employment for IV-A staff with teaching contracts. 

In sum, during 1972-73 a number of major changes occurred in the Atlanta 
School System to disrupt the normal activities of school personnel; thesa 
changes did affect the continuity of school system activities. However, no 
school system is ever static. 

C. THE SCOPE AND STRUCTURE OF THIS STUDY 

The research effort which is the subject of this report began in November 
1972 with the distribution of signal information to AtlanLa school officials. 
The objective of the study was to determine what effect (if any) the pre- 
viously unavailable information on relative performance haa on the manage- 
ment of the school system. The hypothesis which was the subject of 
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investigation can be stated as follows: 

Information on relative performance, when introduced into 
a sufficiently well-understood planning and management structure, 
does (or does not) produce measurable changes (a) in the decision- 
making process of a local school system, (b) in the decisions made 
by school officials, and (c) ultimately, in school performance. 

In its simplest form, the question of whether the signals had an effect 
on school system management involves an analysis of the information feedback 
loop within the school system. Figure III-2 represents that simple feedback 
loop. 

FIGURE III-2 
SIMPLE FEEDBACK LOOP 
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The school systeiTi has an administrative structure which is responsible for 
the operations of the educational process. That process occurs in a set of 
schools and classrooms which comprise the school . system. Signals are de-* 
rived from measures v;hich define the composition of the schools and the 
performance of the students' in ^hose schools. Comparisons are made from 
these measurements and the results analysed and S3mthesi2ed to produce a set 
of signals which are then fed back into the schools' administrative structure. 
In theory^ if school system officials respond to the signals and alter their 
actions, the resulting changes should affect subsequent measurements. 
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comparisons and signals. The cycle is then repeated. 

In an actual school system, of course, the process is not that simple • 
The administrative structure of a large school system involves several layers, 
each with different responsibilities and differing degrees of influence over 
the educational process. Each administrative unit may have need of different 
information to support the decisions it can make. The process of education 
occurs in hundreds of classrooms throughout the system. Each school has a 
different set (or sets) of students and an educational program that may vary 
substantially from grade to grade, A b-:isic purpose of the school system is 
to make changes, over both the long and the short term, in the knowledge and 
behavior of its students. Measurements of educational process, the students, 
and their performance are, at best, imperfect and often non-existent. And, 
instead of one feedback loop, there will be many — to each administrative 
level in the school system, to the school board, to the general public , to 
parents. Each par^y to a decision brings pressure to bear on the educational 
process in response to a variety of incentives. The question under study has 
been the extent to which signals of relative performance became an incentive 
to school system officials to change the educational process. 

Any assessment of the impact of information on the operations of a large, 
complex organization is both difficult and time-consuming. It was necessary 
to limit this one-year research effort to r aageable proportions by focusing 
on those portions of the school system originally thought to have some im- 
mediate, direct impact on the grade or classroom where the measure of relative 

13 

performance is taken. Based on earlier work in Atlanta, nt had become 
13. White, e t al . , op. clt . 
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apparent that the introduction of signal information would not have an equal 
impact on the operations of all parts of the sr'iool system. While it could 
be argued that all school system officials should have an interest in infor- 
mation about relative school performance, not all u- Its within the administra- 
tive structure had an equal opportunity to affect the process of education 
in the classroom in the shcrt time space of this study. 

Two criteria were used to select the administrative units to be studied 
in the effort to discover the .fects of signal information on the school 
system's operations: (1) there must be some direct interaction between the 
administrative unit and the classroom; and (2) the administrative unit must 
have some flexibility to react to signal information. Th^ first criterion 
reflects the fact that the signals are derived from the school and its com- 
ponent grades or classrooms. Each signal reflects the relative level of 
performance of the students in a particular grade at a particular point in 
time, rather than the level of performance of a group of students in an 
entire school or in a special program (e.g., Title I) or special project 
(e.g., after-school study centers). Some administrative units clearly have 
more direct links to the classroom than others. The second criterion, the 
flexibility to act, also stems from the nature of the signals. Signals 
represent clues to the existence of problems and successes at a particular 
point in time. Less than one school year elapsed between the distribution 
of 1972 signals and the production of 1973 signals. Some administrative 
decisions or practices could not be expected to change in so short a period 
of time. 

Applying these two criteria to the'Atlanta administrative structure, 
four units of the school system seemed most likely to be af^ecLed by infor- 
mation .about relative school performance: the Area Superintendents and Lheir 
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staffs, the Divisions of Instruction and Personnel, and the principal. Each 
of these administrative unit 3 has direct and sometimes daily contact with the 
educational process 'at the classroom or grade^ level. These units work singly 
and often in concert to improve that process. Yet, there was lintle evidence 
that the performance of students had been a major consideration iu decisions 
governing most of the activities of any of the administrative units under 
study. 

The principal, the Area Superintendents and their staffs, and the Division 
of Instruction and Personnel engage in a variety of activities which may 
affect the educational process. In its simplest form, the components of 
that process can be thought of as students in a classroom, an instructional 
program and a teacher or teachers. This study considers those activities 
which can affect the process from the outside: (1) changing the teacher 
or teachers; (2) changing the content or the materials used in instruction; 
(3) aug...enting the skills of the teacher(s). Educational^ practitioners 
(and the authors of this report) make the a::.sumption that changes in staff, 
the instructional program or the skills of the staff can improve the per- 
formance of students in the classroom. And, since the level of relative 
performance of a grade in a school is the basis for the signals, the ac- 
tivities of principals, Area Superintendents and their staffs, and the In- 
struction and Personnel Divisions which may change ^he staff or the skills 
of staff or the instructional program in a grade have been the subject of 
this inquiry. In each ciase, the study team attempted to determine how the 
activities are ^'normally" carried on by school officials and hoxv^ signal 
information affected those activities, ' The following paragraphs pose some 
of the questions about* the impact of signals. 
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1. Effects on the Recrultrp.ent , Assignment- Keassignment of Staff , 
Decisions about school staffing involve principals, Area Super- 
intendent Sj and the Personnel Division. Prior to the introduction 
of signals, decisions about the recruiting and placement of new 
teachers did not appear to be influenced by data on student per- 
fonnance in the grade vrhere a vacancy existed. Neither did ntudent 
performance appear to enter into decisions about the reassignment 
of existing staff or the composition of the staff at a school or 

in a grade. Would signal information affect decisions about the 
desired skills or characteristics of new teachers, the placement 
of new teachers, the reassignment of existing staff, the staffing, 
of an entire school? 

2. Effects on the Instructional Program . After consultation with 
teachers, principals. Area Office staff and members of the Instruc- 
tion Division, changes may be made in the strv^cture, content 
materials of the instructional program in a grade. Are signals 
used by any of these parties in decisions about where to make 
changes in the instructional program or in deciding which changes 
to make? Do staff of the Instruction Division or the Area Office 
attempt to associate extremes of relative student performance with 
particular textbooks ,. instructional approaches or organizational 
arrangements? 

3. Effects on Efforts to Improve the Skills of Teachers . Improvement 
in the skills of teachers can be made in several ways: through 
the provision of direct assistance to the teacher by another member 
of the school system's staff or through the teacher ^s participation 
in in-service training programs. Area Resource Teachers, who are 
curriculum specialists assigned to Area Offices, "'regularly provide 
dirpct assistance to classroom teachers and also conduct workshops 
and supervise other in-service activities. The Instruction Division 
also organizes in-service programs. Do signals influence the way 

in which Resource Teachers allocate the time or effort they spend 
in direct assistance to teachers or the decision about v/hich 
teachers to assist? Are signals a factor in decisions by principals 
or Resource Teachers to refer teachers to in-service training 
courses? Are signals used in decisions about the ^jubjects of in- 
service programs? 

Once the administrative units of primary interest to this study were 
identified, efforts were made to determine the extent to which the new infor- 
mation (signals of relative performance) was utilized in the completion or 
assignt'j. responsibilities. In the short run, there c^juld be at least two 
different kinds of impact or effect. The new Information could have had 
a direct effect on the- actions of one or another part of the sc^^ool system. 
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For example, as a result of the signals, a principal might have done some- 
thing to treat the situafion revealed by the signals. But the new information 
might have had little or no direct effect on action and still had an effect 
on the deliberations which precede the decision to act. In this latter case, 
the observed pattern of action will have remained anchanged, but the signals 
might have been consulted by the parties to the action. In the longer run, 
the cumulative effect of using new information in thinking about a course 
of action should be to affect the action taken. Since this study focused 
on a time span of less than one year, it has been concerned with the effects 
cf signal information both on actions taken by school officials and on their 
deliberations , whether or not any action was taken. 

As stated earlier, this project is predicated on the twin assumptions 
that the activities and actors selected for str^idy have some potential for 
affecting the educational performance of students and that those activities 
might be affected by signal information. The truth of the first assumption 
remains to be demonstrated because the interactions described in this study 
are not well documented either in the research literature or, more importantly, 
in the Atlanta School System, In other x/ords, there is neither an accepted, 
recognized level of interaction between any two parts of the school system, 
nor a clear relationship between increases in the quality or quantity of the 
interaction and increases in student performance* 

The absence of any advance knowledge about the level or content of 
"normal" activities selected for study has had important implications for 
testing the effects of signals. First, considerable effort was devoted to 
preparing a description of these routine 'activities , without regard to '.ignal 
information. Second, it was clearly imj-ossible to specify in advancu what 
level of change v/ould occur in any activity after the introduction of signals. 



ERIC 



III-15 



Therefore, the approach taken has been to see Jf grades which received signals 
indicating relatively high or relatively low performance were given dispro- 
portionate attention by school officials. 

The next section of this chapter explains the procedures used to gather 
data during the study. . 

D, PROCEDURES USED IN 'THE STUDY 

Two different techniques were used to gather new data for the study: 
structured interviev^s with participants in the activities under examination 
and special forms or surveys on which Atlanta personnel recorded information 
about specific activities. Efforts to learn about the school level of the 
Atlanta School System focused on a randomly selected sample of 37 elementary 
schools. 

1, STRUCTURED liN^TERVIEWS 

Table III-l (see next page) provides an indication of the scope and 
content of the interviews conducted during the course of this study. Prin- 
cipals, Area Superintendents and Resource Teachers were inter\^iewed on two 
different occasions. 

In several instances, similar questions were asked of the parties to 
an activity or decision which might affect the classroom or grade to provide 
a picture of how each viewed the service rendered or the decision made. In- 
formation about the activities of Resource Teachers, Area Superintendents 
and the Division of Instruction collected during 1972-73 was augmented by 
less structured intervJ.ews in an earlier' phase of this project. 
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TABLE III-l 
STRUCTURED INTERVIEWS 



DATE OF 

irrrERViEW 


PERSONS 
INTERVIEWED 


FOCUS OF li^ERVIEW 


Deceaber 72/ 
January 73 


5 Area Superintendents 


Reactions to signals: ejqjlanation 
of specific signals; anticipa- 


August 73 


4 Area Superintendents 


Actions taken since introduction 
of signals. 


February 73 


31 Resource Teachers 


Reactions to signals; explanation 
of specific signals; descriptions 
of the nature and extent of Re- 
source Teacher contacts with 
teachers, principals, Area Su- 
perintendents, Instruction Di- 
vision staff, and other parts of 
the school syfitem. 


June 73 


18 Resource Teachers 


Whether and how Resource Teachers 
had responded to signal infor- 
mation; review of Resource 
Teacher assistance to classroom 
teachers . 


January - 
February 73 


37 Principals 


Reactions to signals; inf onrLation 
about the general organization 
of the school, its staff, socio- 
economic composition; contacts 
between the school and other 
parts of the school sys teni» 


May 73 


37 Principals 


Specific actions taken as a re- 
sult of signals; services pro- 
vided to the school by Resource 
leacners, xnoLruction uivibion 
staff; changes in instructional 
program and staff for the coming 
school year. 


January - 
August 73 


3 Personnel Staff 


Information about the process of 
staffing schools and the effects 
of signals on that pre, ass. 


May 73 - 


6 Instruction Division 
Staff 


Information about services pro- 
vided to schools and the pro- 
cedures for maki'ig changes in 
curriculum or tcictbooks. 
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2. REPORTED DATA 

Three new data collection instruments were devised and used in this 
•study. Each is described below. 

a. REPORT OF SCHOOL VISIT FORM 

The Report of School Visit form (Figure III-3) was designed for use 
by a Resource Teacher each time he/she visited a school to assist a teacher. 
The form provided the first uniform information about which schools and which 
teachers received assistance from Resource Teachers, as well as other impor- 
tant information abuut the nature of the assistance provided. The Resource 
Teacher i.ndicated on the form the status of the teacher visited (tenured or 
probationary), school and grade visited, the primary subject in which assist- 
ance was given, length of the visit, who initiated the visit, the services 
rendered during the visit and the diagnosis or recommendation made. The 
forms were printed in triplicate, so that two copies could be retained in 
the Area Of^rice for use there. 

The forms were used regularly by Resource Teachers in all five Areas 
from October 1972 until the end of the school year. There were 83 Resource 
Teachers assigned to Area Offices in 1972-73, including both curriculum 
specialists (60) and social workers, psychologists and visiting teachers (23). 
The fomis were intended for use primarily by curriculum specialists. Table 
111-2 provides intormation about the use of the Report of School Visit form 
in each Area. Since no comparable data about this facet of Resource Teacher 
activity existed prior to this time, it is difficult to determine the extent 
t'- which the data provided by Resource Teachers accurately portrays their 
activity. However, the vari::ticn in the total number of forms completed by 
individual Resource Teachers, both within and among the Areas, may indicate 
that Resource Teacher visits to schools were under-reported . 
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FIGURE III-3 
ATLANTA PUBLIC SCHOOLS 

Report of School Visit 
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Primary subject in which assistance was given (Check one) 



.reading oi 
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. language arts 03 



. science 04 

. social studies 05 
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Date of visit . 



foreign language 07 

physical education ca 

vocational education 09 
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discipline 10 
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Place an X on the line below to indicate the approximate length in hours of this visit ( excludin g travel time ). 
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1, Who Initiated this visit? (Check all that apply) 

a. You _d. Area Superintendent 

b. Classroom teacher e. Other (specify: 

J_c. Principal 

2. What services did you render during this visit? (Check all that apply) 
a. Observation of the teacher 

b. Demonstration of teaching technique, materials or equipment ' 

c. Consultation or conference 

d. Assistance v/lih or interpretation of tests 

e. Assistance v/ith extracurricular project 

f. Other (specify: 



Enuxi 



3, What recommendation did you make? (Check ail that apply) 

a. Recommend workshop or in-sor\Mce program 

b. Recommend prcficiency modules 

c. Follow-up visit scheduled for 

d. No follow-up nccebsapy 



. e. Problem r eferred to another Resource Teacher_ 
. f. Other (specify: 



4. Additional comments on the visit. 



URBAN institute: copy 



U.I. 507-1 
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TABLE III-2 

RESOURCE TEACHER RESPONSE TO' REPORT OF SCHOOL VISIT FORI'I 

2Y ARRA. 1972-73 





AREA 
I 


AR£A 
i II 


' AREA 
1 III 


' AREA 
1 IV 


AREA 


TOTAL 


Kuniber cT Resource 
Teachers in Area 


20 


1 20 


1 11 
1 


1 16 
1 


1 16 


83 


Kumber of P^esource* 
Teachers Returning Forn 


13 


1 8 


1 9 
1 


i 12 
i 


^ 10 


52 


Number of Forms Returned* 


672 


' 221 


' 1012 
1 


[ 1010 


266 


3181 


Fewest Forms Cou^pleted'^ 
by Any Resource If .chcr 

Most Forms Con? let ed by 
Any Resource Teacher^ 


6 

177 


! ' 

! 


1 16 

•1 

j 294 


1 2 
1 

1 335 . 
1 


2 
74 


■ 



* Numbers exclude! vir.ltf; reported to .secondary scJiools since 
the enalyscs c£ bow signaJ inf onr.ation nffected Rc?;ourcc 
teacher visits are confined cc those grades for vhich 
signals exiL;tcd (i. e, Grades 1-7), 



b, ' REPORT OF TORKSIIOP OR IN-SERVICE PROGRAM FORM 

The Report of Workshop on In--Service Program form was also designed 
for use by Resource Teachers. It provides information — again, for the first 
time — about the effort Resource Teachers devote to in-service training activ- 
ities during the year. The Report of Workshop or In-Service Program form 
(Figure III-4) was completed by Resourca Teachers each time that they prepared 
and/or conducted a workshop or ia-servx':e program. The Resource Teacher re- 
corded information about who prepared the program, how much time vas spent in 
preparation, who conducted it, the subject and objectives of the program, its 
length of time and who attended it< .Table III-3 provides information about 
the number of workshop forms completed during the 1972-73 school year. 
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FIGURE III-A 
ATLANTA PUBLIC SCHOOLS 



Report of V/orkshop or In-service Program 
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TABLE I I 1-3 

RESOURCE TEACHER RESPONSE TO TORKSHOP OR 
IN-SERVICE FORM, BY AREA, 1972-73 





AREA 
I 


1 AFXA 
II 


1 AREA 
III 


1 AREA 
IV 


1 /JUIA 
1 V 


TOTAL 


No. of forms completed 


70 


1 0 


1 a; 


1 33 


1 14 


204 


No. of Resource Teachers 
cojipleting fonr.s 


12 

1 


0 


f . 


1 


1 7 

1 1 


36 



c. CLASSROOM TEACHER SURVEY 

A mail survey v^as sent to a sample of teachers in 37 elementary 
schools selected for intensive study. The survey was designed to gather 
information about the kinds of services which were actually received by 
classroom teachers in 1972-73. Such information V70uld provide a picture of 
how the teacher perceived the services provided by other parts of the system 
and would help to corroborate information gathered in other parts of the 
study. 

The 37 elementary schools were composed of 199 grade levels or cells. 
Teachers in 96 of those cells were included in the survey: A8 cells were 
selected at random and 48 cells were selected to provide additional infor- 
mation about the services provided to teachers of grades where performance 
was relatively high or low. A total of 233 teachers received questionnaires 
in May 1973; each questionnaire was accompanied by a letter explaining its 
purpose and assuring the respondent's anonymity. 

Table III-4 shows for each Area the number of teachers sampled and the 
number and proportion of questionnaires returned. 
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TABLE III-4 
RESPONSE TO TEACHER QUESTIOMNAIRE , BY AREA 



AREA 


Nunibcr 
Sampled 


Returned 


Percentage. 1 
Returned 


I 


45 


31 


69 


II 


46 


34 


74 


III 


53 


26 


49 


IV 


52 


35 


67 


V 


37 


28 


76 


TOTAL 


233 


154 


66 



The next two chapters of this report describe what was learned as 
a result of these data collection activities. 
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CHAPTEi^ IV 

TllE MAITACEMFNT OF LARGE URBAN SCHOOL SYSTEMS— 
AS OBSERVEIJ IN ATL/JS^TA 

One .f the results of this study has be "^n the accuinulation of knowledge 
about tha actual operations of a large, urban school systen.. This chapter 
presents the information in terms of a discussion of each of the three deci- 
sion areas chosen for study: staffing, the instructic lal program and improv- 
ing the skills of staff. 

A. INTRODUCTION 

This study is concerned with the effects of information on the manage- 
ment of a large school system. It is based on the assumption that actions 
of school officials which directly affect the educational process will ulti- 
mately affect the performance of s.tudents in the classroom. 

The preceding chapter identified the parts of the school system adminis- 
trative structure and the three activities which have been chosen for study. 
The tasks addressed by this chapter are (a) to define for those activities 
how decisions are made and how the various administrative parts fit together 
and (b) to identify the existing bases for making decisions and the incen- 
tives which exist for or against change. An understanding of these tasks 
will, in part, account for the way Atlanta officials responded, or failed 
to respond, to signal information. 

IkiSt of the material in this chapter was collected during the 1972-73 
school year. Each set of activities whica will be discut^sed in the remainder 
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of this chapter is assumed to have a dii.'ect effect on the level of perform- 
ance in the classroom 6t grade — and, thus, an effect on the signals of 
relative performance. 

B. TOE RECRUITMENT, ASSIGNMENT 
AlU) REASSIGNMENT OF STAFF 

There are two basic sources of teachers for any classroom: the teaching 
staff already employed by the school system and the pool of applicants for 
teaching positions. Decisions about which teacher or teachers stand before 
a class may involve the principal, the Area Superintendent an*^ the Personnel 
Division. Personnel Division staff take the lead in recruiting new teachers 
and in making t*he initial assignment of each new teacher; the Area Superin- 
tendent plays the central role in the transfer or reassignment of existing 
teachers. The interactions which occur between the Personnel Division, 
the Area Superintendent and the principal when a new teacher is hired or 
a teaching assignment is changed are described below. 

1. RECRUITMENT OF NEW TEACHERS 

The process of recruiting potential teachers for Atlanta schools operates 
souewhat independently of the actual need for teachers in the classroom. Per- 
sonnel Division staff direct the recruiting drive, supported by a recruitment 
committee composed of representatives from all parts of the school system. 
Most recruiting has centered on college campuses; as a result, most new teach- 
ers hired by Atlanta have been recent college graduates. 
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The annual cycle of hiring and assigning new teachers occurs as follows: 

• Throughout the school year: a file of candidates is created by 
accepting and screening applications from prospective teachers. 
Applicants complete a standard written application form, which is 
supplemented by a personal interview. 

• Late spring and summer: "blanket contracts" are offered to 
candidates based on decisions about the number and type of new 
teachers required for the subsequent school year. 

• Late summer and early fall: vacancies vh.Tch cannot be filled 
with existing teachers are filled by new teachers; additional 
teachers are hired and assigned as needed.. 

Ordinarily, vacancies which have not been filled by the reassignment 
of existing staff are filled by new teachers. The request for a new teacher 
originates with the principal, V7ho iden*:ifi::S the grade and school vjhere the 
vacancy has occurred and the race needed t.o maintain the required racial 
composition of the faculty. The principal sends the request to the Area 
Superintendent, V7ho revievs it. If approved, the request is forwarded to 
Personnel. In addition to information about the location, grade and the 
race of the position, Personnel may have available information p~i the instruc- 
tional approach or organizational* pattern used in the school. While official 
information available to aid placement is limited, principals and Area 
Superintendents do communicate to Personnel information about the vacancy 
on an informal basis. 

Once the assignment is made, Personnel informs the teacher, prlncipr.l 

and Area Superintendent. If the teacher accepts the assignment, the Personnel 

Division considers the vacancy "filled" and does not participate further ia 

the transaction. In only a very fev7 cases is the ne* teacher hired with 

knowledge of the grade and school to which he or she will be assigned. The 

princ'^^pal may receive formal notification, that a new teacher has been assigned 

14. A "blanket contract" ib a contract to teach in the Aclanta School 
System with the specific assignment to .be made later • 

O 
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to the school after the teacher has arrived. Principals have no advance 
information about the cTiaracter istics of 'the new teacher or the teacher's . 
instructional strengths and weaknesses. 

All new teachers are required to participate in an orientation program. 
Half of the program is designed and conducted by the Area Office staff and 
half by the central teacher training staff of the Instruction Division. 

A new teacher is on probation for a period of three years. At the end 
of each of those years, the principal must appraise the teacher's performance 
and make a recommendation about vhecher the teacher is to be retained for 
another year. The Area Superintendent reviews the principalis rating and 
recommendation. Before supporting a recommendation not co retain a pro- 
bationary teacher, the Area Superintendent usually makes an independent 
appraisal of the teacher., eitiier directly or by sending a Resource Teacher 
to observe th teacher's performance. 

Since ^ -tal student enrollment has been declining for the past several 
years, the need for new t:eachers has resulted primarily from resignations 
and retirements of erisuing staff, Resignations are submitted by the teacher 
directly to the Superintendent, with copies to the principal and Area Super- 
intendent. For the 1970-71 school year, an extensive recruiting effort was 
required to fill the large number of vacancies which occurred following the 
court-ordered adjustment of the distribution of black and white teachers at 
each school in the Atlanta system. By ileptember of 1970, approximately 
1,000 new teachers were offered jobs for the school year — nearly double the 
usual number. In September 1972, the school system hired and placed 324 
elementary school teachers. Due to the rapidly declining enrollment and 
the closing of 12 schools, . no new elementary school teachers were hired for 
September 1973. 
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2. ASSIGNMENT OF EXISTING STAFF 

During the course 'of employment by the Atlanta School System, a teacher 
may be :ransf erred to a new grade assignment, transferred to n. new school 
(in the same Area or in another Area) or may be promoted to another position 
within the school system. 

a. TRAIISFERS WITHIN A SCHOOL 

In Atlanta, as in nearly all school systems, the most common and 
basic pool of personnel consists of the staff at an individual school. Once 
assigned to a school, a teacher is under the immediate supervision of the 
principal. Typically, the final assignment of teachers to specific grades, 
classes and tasVs is made by the principal. In particular, principals can 
and do alter grade assignments of teachers in the school from school year to 
school year or, in some cases, during, the school year. These transfers can 
be motivated by any of the following: teacher dissatisfaction with an assign- 
ment; suggestions by the Area Superintendent; decisions to alter the staffing 
pattern and instructional program in the school. The Area Superintendent is 
alv7ays consulted about a change, but Personnel is not involved in these deci- 
sions. The Personnel Division is notified of a change in grade assignments, 
but there may be a considerable time lag. 

b. TRANSFERS AMONG SCHOOLS 

Each spring, the principal and Area Superintendent meet to review 
the school's instructional program, staffing needs and budget for the coming 
year. Based on the student enrollmcat -^'ojected for the following September, 
the staffing allocation for the school is established. The combined staffing 
allocation for all schools in an Area form a somewhat self-contained personnel 
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pool. As long as an Area Superintendent maintains acceptab.'.e student /teacher 
and black/white staff ratios at each school, he can redistribute personnel 
assigned to schools in the Area at his discretion; he need not seek the 
approval of Personnel or of other- Area Superintendents. 

Staff may be transferred into a school for any or all of the following 
reasons: increasing enrollment at the school, resignations oi retirements of 
existing staff; changes in the organization or instructional program at the 
school; special needs of the students; to meet the required racial compo- 
sition of the faculty. At their annual spring staffing conferences, the 
principal and Area Superintendent review the type of personnal needed at the 
school. The Area Supe:. intendent looks first to the pool of teachers within 
his Area in making assignments to a school. It is only when the Area Super- 
intendent cannot fill the positions by transferring staff within his Area 
that the other Area Superintendents and the Personnel Division are called 
Into the staffing process. 

Principals have relatively little input into the decision about which 
teacher (s) will be transferred into their schools. If a principal knows a 
specific teacher is available and that the teacher has the necessary attri- 
butes for the job, the Area Superintendent may help In getting that teacher 
assigned to the school. In most cases, a teacher is transferred into the 
school without the advice or consent of the principal. The principals inter- 
viewed expressed dissatisfaction with the way in which these trancrers take 
place. At present, the formal notice of the arrival of a new teacher comes 
from Personnel; the notice gives only the name and previous school assignment 
of the teacher. Principals expressed their preference for more information 
about the new teacher In advance of the teacher's arrival at the school. 
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Requests for transfer may be sent directly' to the Area Superintendent 
hy the teacher; such reque? ts do not need the approval of the principal, A 
request for transfer is sent on to personnel by the Area Superintendent. 
The teachfir may indicate his/her preferences for r nssignraent • Administra- 
tive transfers (those due to school closin^^s or the ^elimination of positions 
at a school) have priority over requests for reassignment due to personal 
reasons. The Personnel Division takes ai\ active role in the reassignmp.nt 
of teachers only v;hen the transfer involves more than one Area, Then Person- 
nel compiles a list, ordered in terms of seniority, of all teachers requesting 
a transfer and distributes thk-^ list to all Area Superintendents, A *-picher 
who is transferred can appeal the transfer directly to Personnel. 

3. SU^IMARY 

During the past 12 months, four factors have governed the staffing 
decisions of Atlanta officials. Decreasing student enrollment has had 

an obvious impact on staffing. No new teach', s, except special education 
teachers, have been hired since vTanuary 1973; Atlanta now has a huge waiting 
list of applicants for teaching positions. The r losing of 12 elementary 
schools and the conversion of two high schools into middle schools is the 
second factor affecting decisions about staff assignments. Teachers from 
closed schools are given pritjrity in new assignments; high school teachers 
are not always qualified to teach in middle school grades. The third major 
factor affecting sf^.ffxng has been the drastic cutback in federal funds (es- 
pf^iially Title IV-A) . Teachers hired under annual contract^, had to be 
absorbed by the system when federal funds were cut in mid-year. Atlanta 
has not had to ]et any teachers go thus far. The natural attrition of staff 
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uue to retirements and resignations has tt>sc1c the situation somewhat less, 
desperate. The major factor governing staffing assignments for 1973-74 
is tne need to meet the requirements of the courts-ordered racial ratios 
in individual schools. 

C. SHAPING THE INS T RUCTIONAL PROGRA.M 

Decisions affecting the instructional program (the t* cond component of 
the educational process at the classroom level) are also made at several 
levels within the Atlanta system- The details of the instruction program 
for a particular grade or school are the primary concern of the prin:iT^-al 
and staff at the school. The development and revision of curriculum for 
the entire system are a major concern of the Division of Instruction* The 
implementation and day-to-day review of certain facets of the instructional 
program in the classroom fall to the staff of the Area Office. Each unit 
works with the others, even on those activities for v/hich one unit assumes 
primary responsibility. The next two sections describe the shaping of the 
instructional program from the perspective of a- '9f*l joI and of the school 
syc::em as a whole. 

1. SHAPING THE INSTRUCTIONAL PRQGRA>1 
FOR A PARTICULAR GIUjE OR SCKOOL 

Atlanta principals claim major responsibility for decisions about the 
way grades or classrooms are organized and the instructional materials cinc' 
approach used throughout the school. The principal will discuss and s£2k 
approval for changes in' organization or instructional approach with the 
Area Superintendent. Most principals tend to share responsibility for 
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decisions about organizational or curriculum changes with the teaching fac- 
ulty. The choice of textbooks is often made by vote of the entire faculty 
or the faculty of the grade (s) involved. Over the past few years, there has 
been a shift from the traditional self-contained classrooms tov;ard more flex- 
ible teaching arrangements. Once a different crganizacional pattern is 
adopted J Area Re'^ource Teachers are usually called upon to provide assistance 
to the school (principal and teachers) in inplenenting the new pattern. 

The principal and teachers can and do seek advice on instructional 
matters from both the Instruction Division staff and 'ea Resource Teacherr 
The Instruction Division sometimes sends information to the school concerning 
new text offerings in various subject areas; curriculum guides are created 
to assist teachers; information is available about texts and materials; 
periodic workshops are sponsored by the Instruction Division for principals 
and teachers. Principals have indicated that advice on the choice of text- 
books is the most sought after service provided by the Instruction Division* 
While curriculum coordinators from the Instruction Division do visit schools, 
principals interviewed in this study claim overwhelming!' to initiate those 
visits* The two exceptions to this statement are those schools v;her(^ there 
are federal projects (e.g., Follow-Through or Title I) operating and those 
schools piloting the new eleh^pntary curriculum* Direct involvement in the 
instructional program -at a school by Instruction Division staff is more fre- 
quent in these instances* 

A complaint frequently expressed by principals is that there is no 
^'outine way in Atlanta to share information about good practices which ^xist 
in a school or S'-.hools. The principals feel that the existing mechanisms 
for axchanging information about a particular method or text or innovation 
at a school or by a particular teacher are ina^ .equate. The Division of 
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Research and DevelopMnt doe. .*5iiinfliarizc and disseminate information about 
research findings, but the presentation may be too gCi.aral or the results 
not timely enough for :he needs of the principal. While the Area Superin-- 
tendents have regular meetings of -all principals, even these meetings do not 
provide an appropriate forum to exchange ideas for several reasons- First, 
the meetings include principals of all schools — Imary, elementary, middle, 
a^d ai^h schools — and the innovation may be at such a fine level of detail 
that only a fev; of those present wov Id be interested. Perhaps more impor- 
tantly', the agenda for these meetings is often fixed, with n^-i place on it. 
for a general discussion of innovative ideafi or practices. Within the 
schorl, the principal may encourage teachers to observe the classefi of 
colleagi 3S who have demonstrated instructional success. But principals 
repeatedly expressed the need for a regular, systematic way to share the 
information beyond the school level. 

2. SHAPING THE INSTRUCTIONAL PROGRAM 

FOR THE SCHOOL SYSiei . . . . - 

Within the broad outlines of an instructional program for the entire 

syftera, each school artempts to tailor its specific er^ucational program to 

the needs of its stud .nts. The Division of Instruction, which takes the "".rad 

in molding that program* is the largest of the staff groups in the Atlanta' 

structure. Tha development of curriculum and the administration of special 

15 

projects (usually federally funded) are two of its major activities, which 
bear directly on che educat?G^al process in the classroom. 

15. The other broad categories are teacher training (discussed in the 
next section of this chapter) and pupil Shirvices, These categories are 
arbitrary groupings used by the authors of this report. The Instruction 
Division has about two dozen components, each headed by a director ox coor- 
dinator.' It ha3 had no iormal organi?ation .1 structure. 
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Jn the .Atlanta syoten^. tJr,^ deve{ cpPCJi-t: cr re-visioo. ctx curricu\un\ is a 
join-t und«irt.£kk.Ana> ^fa-tKer tha^. tire ev^clusi.ve it"e'Spi:.>>^» ifci 1 i ty of tht, Cnrricuyj.m 
Pevelcpn;t:nt departfi^ent. T[>o LV.ulrc c.V : -.'.nary c.rrf.culuir, is ur:\. r.;.o:n3 i\'vis- 
ion by a conanittee cor.posed of teachers, principals. Area Superintendents and 
Area Resource Teachers, as veil as staff from the Instruction Division. 
According to. a 1970 progress report, the. Elementary Curriculum Revision Com- 
mittee was given the task of developing a curriculum model which '^>'ould enable 
a teacher to begin teaching any child at his performance level in a given area 
and help him move forward." The model was to include: 

6 provisions for individualizing instruction (analysis and diagnosis, 
appropriate content, teaching strategies, and evaluation) 

• provision for the child to perfox^ adequately on his o^m level 
(self-pacing, self-evaluation, and self-direction) 

• plans for continuous progress of students — ' . 

o provision for the development of self-understanding and under- 
standing of others 

The revised curriculum was introduced on a pilot basis at one school 
in 1971-72. During the period covered in this study, the revised curriculum 
was In use at nine more schools. Each school which adopted the new curric- 
ulum was provided with a manual containing Information on hovj to Implement 
the curriculum, teaching aids (such as the teaching model) , as well as 
technical assistance from the Curriculum Revision Committee's staff. In 
addition to technical assistance provided directly from the Instruction 
Division, one Resource Teacher in each Area was responsible for assisting 
the elementary schools piloting the new curriculum and for obtaining equip- 
ment and/or materials needed to implement it. In the administration of 

16. "Elementary Curriculum Revision Commit^tee Progress Report," 
Atlanta Public Schools, -...Summer 197.0, p. l. \ . ^ 
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the school witii the new curriculum, there is a heavy dependence on coTiununlty 
involvement and joint decision making ar.ong principal, teachers and parents. 
The ten pilot schools were selected because it was believed that the prin- 



methrds of organization. 

Another major project of the Instruction Division is the Comprehensive 
Instructional Prograr. (CIP) , a locally ijunded :oTnpensatory education project 
desigpied to improve the basic skills of elementary students. The CIP program 
was begun three years ago. During ite first two years. It concentrated on 
the teaching oT">eading in the primary grades; last year the focus was 
expanded to include the teaching of arithmetic in upper elementary grades. 
Tht two tangible product.^ of the CIP program to date have been the develop- 
ment of diagnostic reading tests for use in identifying and correcting 

reading problems of individual students and the development of proficiency 

17 

modules for use by elementary school teachers of reading and math. Ko 
spe' ial diagnostic test for arithmetic has been developed. 

The merhaniRm for implementing the CIP program has been the assignment 
of 19 Resource Teachers to the five Area Offices to assist principals and 
teachers in improving the teaching of reading and arithmetic. However, once 
Resource Teachers ware assigned to the Area Offices, it was left to the 
i^jrca Superintendent to decide how the Resource Teachers vjould be dit:tributed 
among the schpols in the A.rea. The dirrctor of the CIP program and the 



ci jals, s'.affs and parents would be receptive to the new curriculum and 



Resource Teachers meet weekly to discuss issues of common concern. 



17. Proficiency modules will ba^ discussed iu more detail in the 
teacher training section of this chapter. 
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Staff of the Instruction Division, known as subject-area coordinators 
or specialists, are available to individual schools or teachers for consult- 
ation on curricular matt'-iis. However, most of the curriculum coordinators 
are oriented toward middle and high schools. One way the Instruction Divi- 
sion does have a direct effect on elementary schools is in the selection 
and purchase, of textbooks. Most textbooks are paid for v7ith state monies 
and their adoption is subjecv to state approval. According to state guide- 
lines, a textbook may have a life of no inore than five years. Once every 
five years, the state reviews all texts in a given subject or discipline 
and produces a list of textbook series that can be used in the state of 
Georgia, Following the publication of one of these lists, .the Atlanta Text- 
book Coordinator establishes a committee of teachers, Resource Teachers, 
and Instruction Division staff to review and raue texts on the basis of 
their appropr ateness for use v;ith students performing at various levels. 
The results of these reviews (abo- t a paragraph on each text series) are 
distributed to the principal as a basis for ''electing text series for use 
in the school. Typically, one text series in a given discipline is adopted 
for an entire school, 

3, SUMMAR Y 

Decisions about the shape and i. intent of the instructional program are 
probably the best example of the flild management which characterizes the 
operation of Atlanta schools, Eac'.i :Lnstructional decision usually involves 
one or more representatives of each administrative level in the Atlanta 
system. Educational goals are vaguely worded. The outlines of the Instruc- 
tional programs to meet ':hose goals are filled in under the direction of 
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the Instruction Division staff. This staff, however, spends relatively 
little time actually in the classroom. Instead, the Instruction Division 
looVs to Area Resource Teachers to make operational tho^e guidelines. There 
appears to be a lack of any clear 'delineation of the responi^ibilities 
of Area SuperintendentsJ and their staffs on the one hand and the Instruction 
Division on the other, for the implementation of the instructional program. 
As a result, the principal and teachers at a school make the crucial deci- 
sions about the actual v/ay in which the curriculum is assembled and imple- 
mented • No well-defined feedback mechanism exists for staff outside the school 
to determine what kind of instructional program really exists in a classroom 
and what problems the teacher may encounter in implementing the curriculum. 

D. niPRQVING THE S KILLS 
OF TIIE TEACHING STAFF 

A third set of actions v:hich affect the eaacational process in the 
classroom are those which attempt to improve the skills of the teaching staff • 
Two kinds of actions will be considered: (1) the provision of direct assist- 
ance to classroom teachers, which usually occurs on a one-LO-one basis; and 
(2) the participation by the teacher: in one of the structured teacher train- 
ing activities available in Atlanta. 

1, DIRECT ASSISTANCE TO TEACHERS 

Classroom teachers can and do Initiate requests for direct assistance. 
Often they turn to colleagues on the staff or to the principal. However, 
because the principal and the teacher have daily contact within the school, 
the principal is often the person who first becomes aware that a classroom 
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teacher needs assistance. According to principals interviewed in this 
study, a major clue to a teacher's performance comes from the principal's 
personal observation of the teacher. Most principals report several visits 
a week to each classroom to check on both teacher and student progress. 
Casual visits to the classroom are augmented by periodic conference?.? between 
th principal and teacher. The principal may also demonstrate a lesson for 
a teacher or teachers; 40 percent of the principals interviewed reported 
giving demonstration lessons on such topics reading, how to use diagnos- 
tic tests, science and story telling. : 

Outside the school, the most }:eadily available source of assistance 
for the classroom teacher is the Area Resource Teacher. Sixty of the 83 
Area Resource Teachers are curriculum specialists', the number of curriculum 
specialists assigned to any one Area varies from 6 to 15. Math, science, 
English, foreign language and social studies Resource Teachers are assigned 
to work primarily vzith high scliool L^ac'^rs and principals. Art, music and 
library Resi urce Teachers work with art or music teachers or librarians 
assigned to the schools In the Area, and only infrequently vith classroom 
teachers directly. The remainder of the curricula specialists in each 
Area work almost exclusively with elementary schools. 

The way in which the Resource Teachers are utilized varies from Area 
^o Area. Some Area Superintendents assign each elementary Resource Teacher 
to a group of schools (although the number of schools assigned varies from 
Resource Teacher to Resource Teacher, even vjithin the same area). Decisions 
about which- teachers to assist, what services to provide or the conditions 
under which a visit is made are usually left to the discretion of the 
Resource Teacher. 
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During 1972-73^ Resource Teachers reported 3181 visits to elementary 
or middle schools. Each time a visit was made, the Resource Teacher was 
asked to indicate ^.''ho initiated the visit: the Resource Teacher, the teacher, 
the principal, the Area Superintendent. Table IV-l below presents the respon- 
ses to that question [laultiple responses were permit ted ] , tabulated by Area. 



TA13LE IV- 1 

INITIATORS OF RESOURCE TEAG^R VISITS, 1972-7 3* 



Initiator 


Total 
(n-3181) 


Area I 
(n=672) 


Area 1."'. 
(n=221) 


Area III 
(n=1012) 


Area IV 
(n=1010) 


Area V 
(n=266) 


Resource 
Teacher 




63% 


69% 


49% 


11% 


68% 


Teacher 


23% 


30% 


23% 


20% 


20% 


117. 


Principal 


19% 


23% 


11% 


22% 


16% 


19% 


Area Super- 
intendent 


12% 


27% 




9% 


11% 


0.4% 



'^"Source: Report of School Visit Forms 



According to the information provided by Resource Teachers, half to three- 
quarters of all visits to elementary or middle schools are initiated at 
least in part by Resource Teachers. Teachers are credited with Initiating 
visits more often than principals. 

Resource Teachers may provide assistance to teachers in a variety of 
curricular or instructional areas. During 1972-73, Resource Teachers were 
asked to indicate, for each visit reported, the primary su.bject of the 
assistance given. Twelve options were piTOvided : nine related to curricular 
areas; the other three vi^eie planning, organization a.id discipline. Although 
asked to indicate only one subject for each visit, most Resource Teachers 

r 

18. See Chapter m p. 17 ff. 
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reported providing assistance on several topics during a single visit. 
Table IV-2 below sumtnarizes data on the subject of the assistarce reported 
during 1972-73. 



TABLE IV-2 

SUBJECT OF THE ASSISTANCE GIVEls 
BY RESOURCE TEACHERS, 1972-73 "' 



SUBJECT OF 
ASSISTAXCE 


Total 

(u=3181) 


Area I 
(n-67Z) 


Area II 
(n=221) 


Area III 
(n=].012) 


Area IV 
(n=1010) 


Area V 
(n=266) 


Reading/ 
Language Arts 


47% 


64% 


29% 


24% 


55% 


82% 


Arithmetic 


10% 


13% 


1% 


4% 


14% 


16% 


Social Studies 


5% 


2% 


13% 


8% 


3% 


3% 


Science 


A% 




2% 


2% 


5% 


2% 


Planning/ 
Organization 


36% 


44% 


16% 


42% 


27% 


43% 


Discipline 


3% 


6% 


3% 


2% 


3% 


4% 



* Source: Report of School Visit Forms 



Reading and language arts have been the most frequent subject of Resource 
Teacher visits, except in Area III, The frequency with v.'hich assistanne was 
provided in other subject areas drops off sharply. Planning and organiza- 
tion seem to have been the other major concern curing 1972-73. 

An examination of data on the nunber of schools and grades visited at 
least once during the year provides another insight into the pattern of 
assistance provided to classroom teachers. Based on reports filed by 
Resource Teachers, at least one classroom teacher (K-7) was visited in all 
but four of the 127 elementary and middle schools during 1972-73; in addi-- 
tion, Resource Teachers reported visits to principals and other staff not 
assigned directly to the classroom (e.g., music teachers) in more than 85 
percent of the schools* 
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The number of grades '^Isited at least once during 1972-73 is shown in 
Table IV-3 below. The -'iata are presente4 by Area, first for Grades 1-7 and 
then for kindergarten. Based on reports filed by Resource Teachers, teachers 
in 62 percent of the 946 grades in Atlanta elementary and middle schools 
were visited at. least once during the year. There was considerable varia- 
tion among the Areas in the coverage reported by Resource Teachers, 

TABLE IV- 3 

PROPORTION OF GR^aDES 
VISITED BY RESOURCE TEACHERS, 1972-73>'^ 



Area 


Grades 
Total 1% 


1-7 

Visited 


Kindergarten 
Total 1% Visited 


Grades 
Total 1% 


K-7 
Visited 


I 


126 1 


75% 


19 1 


89% 


145 1 


77% 


II 


1 

154 1 


41% 


22 I 


77% 


176 1 


45% 


III 


179 1 


62% 


1 


44% 


206 1 


60% 


IV 


189 j 


71% ■ 


" 1 


33% 


216 1 


67% 


r 


175 1 


58% 


28 , 


11% 


203 1 


51% 

1 


Total 


823 1 


61% 


123 1 


47% 


946 1 


60% 



'-^Source: Report of School Visit Forms 



Perhaps a better indication of the extent of the direct assistance 
provided by Resource Teachers can be seen in Table IV-4 below. The table 
shows the frequency with which Resource Teachers visited individual grades 
(l--?) in individual schools, ^fliile frequency does not say anything about 
the length of the visits or the nature of the assistance provided, it does 
indicate the amount of exposure Atlanta teachers had to Resource Teacher 
assistance in the classroom. Thirty percent of 506 grades (1-7) visited 
were visit'^d only once during the year; 15 percent were visited only twice 
by Resource Teachers; just 8 percent of the grades visited were visited 
more than ten times. The pattern of visits by Area IV Resource Teachers 
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TABLE IV-4 

FREQUENCY OF VISxTS TO GRADES 1-7 
BY RESOURCE TEACHERS, 1972-73* 



Percent of Grades 1-7 Visited 



Frequencj- of 


Total 


Area I 


Area II 


Area III 


Area IV 


Area V 


Visits 


(n=506) 


(n=95) 


Cn=53) 


(n=112) 


(n=135) 


(n=101) 


1 


2 9% 


29% 


48% 


31% 


14% 


36% 


2 


15 


8 


21 


16 


11 


20 




1 9 




1 1 

XX 


12 . 


14 


X J 


4 


10 


13 


2 


10 


10 


12 


5 


G 


7 


10 


4 


7 


•4 


6 


6 


4 


6 


5 


6 


8 


7 


4 


2 


3 


5 


7 


4 


8 


3 


6 


0 


4 


4 


2 


9 


3 


7 


9 


0 


6 


1 


10 


2 


2 


0 


4 


4 


0 


Over 10 


8 


9 


0 


8 


17 


1 




* Source: 


Pvepor t 


of School 


Visit Forms 





is sordev7hat different. In that Area, 17 percent of the grades visited were 
visited on uore than ten occasions, and only 14 percent of the grades visited 
were visited only once. 

The pattern of assistance to classroom teachers varies from Resource 
Teacher to Resource Teacher. Table IV-5 (see next page) presents informa- 
tion about the visits reported by each of the 52 Resource Teschers who 
supplied information for 1972-73. The designations I-A, I-B, etc., Identify 
individual Resource Teachers who filed reports. Column 1 indicates the 
total number of elementary and middle schools visited by that person, either 
to aid a classroom teacher or to visit v/ith some other member of the school's 
staff. Column 2 indicates the number of those schools for which at least 
one report of assistance to a teacher in Grades 1-7 was made. Column 3 shows 
the number of different teachers (in Grades 1-7) assisted by each Resource 
Teacher during the year. 
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A^ssiri?;.*) B--; Kfi':ajr^cB za-.cyjLt^ .i97:-75* 



RESOURCE 
TEACiiER 

li L .,Z^V 






CO;..l'MN 3 


TOTAL 
OF SCHOOLS 

visriiiD 


scr.c-OLS i-TjirrC • 

ASSISTANCE V.\S 
PKUVIDri) TO irACIIKRS 
GRAjF.S' 1-7 


TOTAL VlTfBKR OF 
THACi:n?:S ASSISTED 
III GK:\DES 1-7 


I-A 
I-B 
I-C 
I-D 
I-E 
I-F 

1— o 

I-U 
I-I 
I-J 
I-K 
I-L 
I-M 


18 
15 
8 
4 
5 
2 
Ix 
12 
6 
7 
5 
10 
3 


6 
12 
0 
0 
5 
2 

3 
4 
5 
0 

. 0 
3 


7 
19 
0 
0 
39 
31 

^0 

5 
5 

34 
0 
0 

17 


II-A 
II-B 

TT P 

II-C, 

II-D 
II-E 
II-K 
II-G 
II-K 


4 
11 
8 
5 

19 . 
5 

19 
4 


4 
2 

7 

4 
19 

A 

• 3 
0 


• 4 
4 

1 1 

XI 

12 
47 
18 
3 
0 


III-A 
IIl-B 
III-C 
II M 
III-E 
III-F 
TTT-fi 

III-H 
III-I 


27- 
28 
22 
22 
26 
13 

24 
11 


25 
15 
10 
11 
19 
4 

V 

13 
1 


72 
44 
46 
20 
48 
6 

- 21 
1 


IV-A 
IV-B 
IV-C 
IV-IJ 
IV-E 
IV-F 
IV-G 
IV-II 
IV-I 
IV-J 
IV-K 
IV-L 


14 
8 
19 

10 * 

5 

8 

2 

5 

1 

3 
12 

2 


13 
1 

16 

10 
5 
7 
2 
5 
0 
3 
0 

' 0 


28 

1 
35 
79 
• 45 
82 

4 
24 

0 

8 

0 

0 • 


V-A . 

V-B 

V-C 

V-D ' 

V-E 

V-F 

V-G 

V-H 

V-I 

V-J 


5 
A 
9 
4 
2 
4 
13 
4 
4 

2 . 


3 
4 
2 

4 . 
' 2 
4 
9 
4 
4 
0 


7 
14 

2 
11 

2 
44 
35 
26 
39 

0 



* Source: Report of School Visit Forrts Conplcted Durinc 1972-73. 
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As can be seen in Table IV-5, the number or schools visited by a single 
Resource Teacher ranged from 29 to 1; the number of teachers in Grades 1-7 
assisted by a single Resource Teacher ranged from 82 to 0, T\^o-thirds of 
the 52 Resource Teachers reporting visited at least one \school in vhich no 
classroom teacher \:as assisted. Variations within Areas are as great as 
variation betv;Gen Areas. 

19 

The reports filed by Resource Teachers accounted for 3,625 hours 
spent visiting the schools. Table IV-6 shows the total amount of time each 
Resource Teacher reported (Col. 1) and the amount of time spent v/ith class- 
room teachers in Grades 1-7 (Col. 2). The final column in Table IV-6 shows 
the proportion of reported tiirie spent vjith teachers in Grades 1-7. The 
remaining portion of each Resource Teacher's reported time was spent with 
principals, kindergarten teachers and other staff not directly assigned to a 
grade. Although 52 Resource Teachers reported visits to schools, ten of 
them V7orked exclusively with school staff not assigned directly to Grades 1-7. 

In summary > according to the reports filed by Resource Teachers, the 
direct assistance provided by tnem during 1972-73 tended to be primarily self- 
initiated. Reading/language arts and planning/organization were the primary 
subjects of assistance rendered. Resource Teachers visited teachers in 61 
percent of the elementary grades at least once during the year. Fifty-six 
percent of the grades which were visited at all by Resource Teachers were 
visited three times or less during the year. The number of schools visited ♦ 
the number of teachers assisted, the amount of time spent in direct assist- 
ance varies from Resource Teacher to Resource Teacher. 

r 

19. This total probably underrepresents the time spent at schools 
for three reasons: the forms were not used during the first month of school; 
Resource Teachers do not a])pear to have reported every visit to a school; no 
time was reported on approximately 3 percent of the completed forms. 
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TABLE IV- 6 

TIME SPENT AT SCliQOLS VISITED 
BY KESOUFCL^ TKACHrA, 1972-73* 



r> T* C m f r» t* 

RESOU KCL 
TEACHER 
BY AREA 


coi.u::n i 


COLl^'-N 2 


COLUMN 3 


TOTAL TIME (I:; HOL'RS) 


TOTAL TIME (IN HO'JRS) 


PERCENT OK TOTAL 


REPORTED SPENT 
AT SC\'O0LS 


REPORTED SPENT 
IN GRADES 1-7 


REPORTED TIME 
SPENT IN GR.'vI)ES 1-7 


I-A 


1 /. 0 

1m o 




1 Q 


T D 






An 


T 


/J 


n 


Q 


T _ ■n 
i~U 


/ U 


n 
U 


ft 


i— L 


1 P. /• 


± OD 




I-F 


22 


22 


100 


I-G 


89 


54 


61 


i— n 


00 


D 


1 n 


i— i 


Q 


D 




i — J 


1 OR 


Qn 






D 
O 


n 


n 


i-L 


An 


n 
U 


ft 




£^ 




TOO 


II-A . 


4 


4 


100 


7 T — Ti 
1 l—D 


AA 


0 
0 


1 R 


II-C 


56 


49 


87 


iVd 


2A 


20 


85 


II-E 


64 


60 


93 


II-E 


32 


26 


82 


II-G 


40 


3 


8 


II-H 


15 


0 


0 


III-A 


124 


75 


60 


III-B 


302 


39 


13 


Ill-C 


150 


83 


55 


III-D 


77 


40 


52. 


III-E 


160 


69 


43 


T T T — IT 


MO • 






III-G 


55 


0 


0 


Ill-Il 


86 


52 


60 


III-I 


25 


2 


8 


IV- A 


60 


58 


97 


IV-B 


14 


0.5 


4 


IV~C 


101 


83 


82 


IV-D 


313 


267 


85 


IV-E 


107 


98 


91 


IV-F 


376 


339 


90 


IV-G 


7 


7 


100 


IV-H 


36 


36 


100 


IV-1 


6 


0 


0 


IV-J 


36 


' 32 


90 


IV-K 


46 


0 


0 


IV-L 


2 


0 


0 


V-A 


7 


6 


86 


V-B 


22 


22 


100 


V-C 


32 


8 


25 


V-D 


15 


13 


87 


V-E 


2 


r 2 


100 


V-F 


70 


67 


96 


V-G 


110 


66 


60 


V-H 


39 


38 


97 


V~I 


76 


62 


82 


V-J 


2 


0 


0 



* Source; Report of Schooi Visit Forms Completed During 1972-73 
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2. TEACHER TRAINING ACTIVITIES , 

Another approach to improving the skills of the classroom teacher, 
teacher training, may be defined as the set of organized and structured 
courses, \'orkshops and other enrichment activities available to Atlanta 
teachers. Participation in one or another of the teacher training activities 
almost alvzays occurs as a result of a decision by the teacher. A teacher may 
(:nroll in teacher training activities for a variety of reasons: to earn a 
salary increment; to improve general skills; to learn new techniques; to 
obtain an advanced degree; to enhance the possibility of proraotion vithin 
the school system's hierarchy. VThile a principal, a Resource TeacV>*^.r or 
other personnel nay suggest that a teacher enroll in a training program, no 
one may compel a teacher to attend. 

There are four major types of teacher training cpportunities offered to 
Atlanta teachers: university courses outside the Atlanta school system; 
programs sponsored by the Instruction Division; workshops conducted by Area 
Resource Teachers; and proficiency modules. Each of these teacher training 
activities will be described below* . 

a. UNIVERSITY COURSES OUTSIDE THE 
ATLANTA SCHOOL SYSTEM 

A teacher may enroll in a university course for credit tov/ards a 
salary increment or to obttin an advanced degree. The Atlanta Area Teacher 
Education Service (MTES) , founded in 1945, publishes each year a list of 
courses offered at local universities which have been approved for salary 
increment credit by the school system. 

Very little is kno\m about which specific teachers are enrolled in such 
courses. Several principals interviewed indicated that teachers at their 
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schools were enroiled in these courses. When a teacher successfully com- 
pletes a course, the school system is notliied. The Personnel Division 
maintains the payroll records for the school system and is notified by the 
Instruction Division when a teacher has completiid sufficient training to be 
entitled to a salary increase, 

b, INSTRUCTION DIVISION PROGllAMS 

The Instruction Division sponsors a variety of courses, conferences 
and workshops for Atlanta staff, including teachers. This Division is also 
responsible for the orientation program which each new teacher in Atlanta is 
required to complete. 

Training activities sponsored by the Instruction Division usually have 
salary increment credit associated with them and are Imown in Atlanta simply 
as "in-service" activities. At the beginning of each school year, a list of 
the courses or workshops to be offered during the year is distributed to 
Atlanta staff. The list of workshops or in-service programs is amended 
throughout the year, as new courses are offered. 

Final data were not available on participation in 1972-73 programs 
sponsored by the Instruction Division. There V7as no indication, however, 
that the pattern of participation would differ substantially from that for 
the preceding school year. According to the school system's data, 57 
approved courses were offered by the Instruction Division in 1971-72. The 
courses varied in subject matter , length and the kind of participants who 
enrolled. Subjects relevant to elementary teachers included are art (2 
courses), communications skills (1 course), English (2 courses), mathematics 
(2 courses), science (3 courses) and social studies (7 courses). 
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A total of 1,507 staff members participated in those courses. Atlanta 
records do not indicate how many different teachers were involved in these 
in-service programs. The school system does not maintain permanent or 
historical records either on which staff members have received in-service 
training in a given subject or what the training may have involved. At no 
time are principals officially or systematically informed which teachers are 
enrolled in an in-service program. The teacher usually informs the principal 
directly, but no mechanism exists to give the principal or Area Superinten- 
dent a record of V7hich teachers are enrolled. 

The Instruction Division is also responsible for organizing an orien- 
tation program for each new Atlanta teacher. The Instruction Division 
establishes guidelines for the program which require the new teacher to 
attend special meetings, to observe an experienced teacher and to visit 
local points of interest. Each Area Office tailors the contents of one- 
half the orientation program to meet the needs of the Area. 

c. AREA WORKSHOPS . ' 

Resource Teachers conduct workshops for teachers and other staff 

of the schools in each Area. During 1972-73, Resource Teachers in four of 

the five Areas provided inf rmation about the workshops conducted through 

the Area Office. Information on 204 in-service programs was provided by 36 

different Resource Teachers on the Report of VJorkshop or In-Service Program 
20 

form. No forms were received from Area II Resource Teachers. 

Workshops were offered on a variety of topics: reading, art, team 

teaching, classroom management and testing, to name but a few of the subjects. 

Reading (or language arts) was named in more than 25 percent of the forms as 
20. See Chapter III, p. 19 for a description of the form. 
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the topic of the workshop. No other topic was so often the subject of an 
Area workshop. Only 6 percent of the workshops dealt with arithmetic, 
although 15 percent of the teachers surveyed in conjunction with this pro- 
ject indicated a desire for in-service programs in arithmetic. Approximately 
10 percent of workshops deal with classroom organization or management 
or the planning of the instructional program. 

A typical Area V7orkshop is organized and conducted by Area Office staff 
for the benefit of all teachers in the Area. However, more than one-third 
of the workshops were held at individual schools and were attended exclu- 
sively by the teachers at the school where the workshop was held. In one 
Area, 11 workshops were organized for the faculty at a newly opened school. 

More than half the workshops were attended by more than ten persons; 
attendance ranged from a high of 100 to a low of four persons. The typical 
program was conducted after school hours and lasted about two hours. Nearly 
three-fifths of the workshops were completed in a single session; the 
remainder were continued over several sessions. Some Area workshops spanned 
several calendar months and met on a weekly basis. 

d. PROFICIENCY MODULES 

Proficiency modules represent a new direction in in-service training: 
a series of structured individualized instruction for teachers. The modules 
were developed several years ago by the school. system and the University of 
Georgia, as a part of the Comprehensive Instructional Program (CIP), To date, 
modules have, been completed in reading and arithmetic. A module is designed 
so that a teacher takes a test to determine proficiency in teaching math or 
reading. The results of the test determine weaknesses of the teacher, who 
then may choose one of several learning paths to gain needed proficiency. 
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Each teacher progresses through the module at his/her own rate, completing 
one topic at a time. The School Board in late December 1972 adopted a 
policy that probationary teachers who do not satisfactorily demonstrate pro- 
ficiency in teaching must complete at least one module before being advanced 
to tenured status. Implementation of this policy is only just beginning. 

Successful completion of a set of proficiency riodules carries with it 
salary increment credit, and the overall supervision of modules rests with 
the Instruction Division. Day-to-day management of the modules falls to the 
Area Office; Resource Teachers assist classroom teachers with the modules by 
providing needed materials and by being available at specified '^imes for 
assistance. The classroom teacher must pass a written test on the contents 
of each module unit and then Successfully demons';rate the newly acquired 
proficiency in a classroom situation. 

• Each Area Office was asked to provide a list of participants in the 
modules and to indicate which teachers completed the module program. Area 
III does not use the modules, but the other four Areas provided some infor- 
mation about participation. In Areas II and IV, a total of 42 teachers 
enrolled in the module program; these Area Offices did not Indicate how many 
teachers completed the modules. In Areas I and V, a ^'Otal of 66 teachers 
enrolled in the modules, but only 15 completed the module program. 
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E. CONCLUSION 

This chapter has focused on three areas of the Atlanta school system 
that affect the educational process as it occurs in the classroom. Changes 
in staffing assignments, in the instructional programs, and in the skills of 
classroom teachers have been discussed in terms of the decisions that are 
made, the actions that result and the actors who influence them. 

The process described is not a systematic one; decisions are made in 
response to a variety of pressures. Actions are often taken on a case-by- 
case or ad hoc basis. Information about the performance of students rarely 
enters the decision process. 

The next chapter describes the effects observed when information on 
school performance was introduced during the 1972-73 school year. Partic- 
ular ejnphasis is placed on effects of signals on the three activities 
described in this chapter. 



CR^PTER V 

EFFECTS OF THE^ INTRODUCTION OF SIGNALS 

This chapter presents the reactions of Atlanta personnel to the idea 
of comparing relative performance and then' discusses the impact of signal 
information, on the three types of activities discussed in the previous chapter: 
staffing, the instructional program and Improving the skills of the staff. 

A. INTRODUCTION 

At the outset of the current study, two different kinds of signal 
' 21 

materials were distributed to Atlanta staff: (1) booklets containing a 
brief explanation of the derivation of signals, the signals and mean achieve-: 
ment scores for each school in an Area; (2) sheets displaying the signals 
for all schools in an Area for each of two years (1971 and 1972). A smf)ller 
booklet was prepared for each school principal containing the explanation of 
the derivation of signals and the signals and mean achievement scores for 
his/her school. 

The distribution of these materials began in November 1972. Signal 
booklets were never distributed directly to teachers, since the signals were 
viewed primarily as a management tool for use by school officials above the 
school level. The extent to which signals were made available to teachers 
was left to the discretion of the principal. [Principals of six schools 
requested and received additional booklets for distribution to teachers.] 

21. See Figure II-4, p. 11-13. 
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Signal materials were distributed to principals and Area Resource Teachers 
through a series of briefings organized by each Area Superintendent. Signal • 
materials were explained and distributed to Instruction Division and Person- 
nel Division staff in December 19-72 and January 1973. 

The effects of the introduction of signals in Atlanta will be discussed 
in the next two sections of this chapter. The idea of signaling relative 
performance was generally favorably received. Since the impact of 
signal information is partly a function of how the information is received, 
the next section of this chapter describes the reactions of school system 
officials to the idea of signaling relative performance. Section C turns 
to an examination of the impact of signals on the activities described in 
the preceding chapter. 

B. REACTIONS TO SIGNALS AMONG 
SCHOOL OFFICIALS 

The reaction to signals of relative performance throughout all layers 
of the Atlanta administrative structure was generally positive. There was 

widespread praise for the format used to present a massive amount of data 

♦ 

in a clear, concise manner. Two Area Superintendents noted that previously 
they had never had time to examine student performance as measured by 
achievement test results because of the volume of data involved. The com- 
puter print-out. of achievement tx^st results for the schools in a single 
Area is usually several inches thick • Principals also commended the format 
and the handy size of the signal booklets. 
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Reactions to the idea. of comparing relative performance of schools were 
somewhat more varied. Atlanta staff at aril levels endorsed, in principle,, 
the comparisons of schools which have similar student populations. 
Especially appealing to some principals was the idea of comparing performance 
among Atlanta schools, rather than with the national norm established by the . 
achievement test manufacturers. 

The same ambivalence toward the use of achievement test results to judge 

performance which has been expressed by the larger community of educators was 

echoed by Atlanta officials, l^ile admitting their use of achievement 

results to confirm opinions about teachers, curriculum or students, school 

officials (especially principals) expressed concern over the reliability and 

appropriateness* of achievement tests as an evaluative tool. 

The signaling technique relies on participation in the free and reduced- 
22 

price lunch program as the criterion for identifying similar schools. 
Criticism of the use of subsidized lunch participation in generating signals 
came almost exclusively from a few principals and Resource Teachers. 

They expressed reservations about the use of free and reduced-price 
lunch data to describe the socioeconomic composition of a school. Because 
the income cut-off point for eligibility to participate in the program is so 
low, principals of some schools where small numbers of students receive sub- 
sidized lunches felt that the variable overstated the economic composition 
of the school. That is, two schools — one composed of the children of corpora- 
tion presidents and the other, children of assembly line workers — may both 
have comparable rates of participation In the subsidized lunch program, but 

would not be expected to have similar patterns of achievement. 

22. See Chapter II, p. 7 for the discussion of how the variable is 
used in conjunction with mean achievement to generate signals of relative 
performance. 



SoYr.e. rcc i -J/-: ^rrits Sv^t^t^^S iug^cstedi fir.at: ^- because of the ccn^^o»vc.rs]/ 
ever sajNDol dese^^x^GA^tif I ar.d tVx, urnsu^l degree cK* student ;voc il:lM <5uc. t-o 
' as yet uncert^^in busir^s patbcrrsSj the vaciabl-o ^as not: sctLtablc xit the 
present tir.e; thccc officials urually conceded that subsidized lunch partici- 
pation \;Ould be a saticfactcry socioecononic indicator xchen the situation 
becomes more stable. Suggestions for other variables to use in lieu of or 
in addition to free and reduced-price lunch participation included neigh- 
borhood stability, teacher and/or pupil attendance data, community involve- 
ment, parental interest in the school, and evaluations of teacher . performance. 
No one interviewed could offer a satisfactory source of data presently avail- 
able in Atlanta for measuring these attributes. 

23 

In the initial round of interviews, -Area Superintendents, Resource 

Teachers and principals were asked to account for the existence of particular 

24 

red or blue signals. In the second round of interviews, these officials 
were questioned about how signals had been used or who in the school system 
Could have most use for the information. 

• Few principals expressed surprise at the signals for their schools; 
this reaction lends operational support, to the statistical evidence of the 
validity of the signals. The presence of a blue signal in a grade was 
generally attributed by the principals to the success of a teacher or 
teachers. The presence of a red signal was generally attributed to student 
discipline problems, student transiency, overcrov/ding, the absence and/or 
inexperience of particular teachers. Principals had more difficulty 

23* See Chapter III, p. 15, for a description of these interviews. 

24. The reader is reminded that a red signal (either full or half ' 
red) indicates relatively lov; student performance and a blue signal (either 
full or half blue) indicates relatively high performance. Most grades 
receive a neutral signal indicating no extreme of performance. (See Chapter 

II.) 

ERIC 
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accounting for situations In which the signals for the same grade in math 
or reading were different. Many principals wanted to know why certain grades * 
were signaled and not others; they felt that explaining the reasons for 
signalt^ would lead to the development of solutions to problem situations. 

Principals were very inconsistent in their responses to questions about 
the usefulness of signal ' :formation. Nearly 80 percent of the principals 
interviewed felt that signals could be of most use to principals t.nd teachers, 
while less than half felt that Area personnel could make best use of signal 
information. When questioned about the routine interactions between 
principal and teacher, few principals felt signals would have any effect on 
those interactions. The principals did suggest that signals could affect 
the relationship between the school and the Area Office. They predicted 
that the signals might cause both the Area Superintendent and Resource 
Teachers to pay more attention to grades with red or blue signals, that 
Resource Teachers might visit those grades more often, that Area staff might 
set up workshops to deal with problems suggested by patterns in the signals. 
Principals were less certain about the effects of signal informaticn on the 
activities of the Instruction Division or the Personnel Di^^ision. 

Resource Teachers and Area Superintendents had more difficulty in 
interpreting signals for specific schools and grades because the impressions 
on which they base their judgments of performance are not equally current for 
all schools. The presence of a red or bDue signal tended to be explained in 
terms of the ''quality of teaching" in that grade or the overall "quality" of 
the faculty at the school. Other explanations for signals — either red or 
blue — included curriculum changes, test administration, teacher attitudes, 
and the presence of "unusual" students at the school* 
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Both Area Superintendents and Resource Teachers tended to focus on red 
signals. Several Resource Teachers plann-ed to examine the reading programs 
in grades with red signals In reading or in schools with several red signals. 
One Area Superln::endent was so surprised by the red signals at tx^70 schools 
In the Area that he changed the assignment of one of the reading Resource 
Teachers to include those schools. 

In summary, the signals of relative performance distributed for the 
first time in Atlanta in late 1972 were well-received, particularly by 
principals, Area Superintendents and Resource Teachers. The underlying 
principle of comparing performance in similar schools was accepted. The 
method of displaying signal information won praise for i^s gimpliclty. The 
transition from-accepting the idea in principle to the application of signal 
information to decisions by school officials was much more difficult to 
make. The remainder of this chapter discusses the effects (or lack of 
effects) of signals on (1) ,"he assignment of teachers and the staffing of 
schools; (2) the shaping of the instructional program; and (3) improving 
the skills of the teaching staff. 

C. IMPACT OF SIGNALS ON TEACHER ASSIGNMENT 
AND SCHOOL STAFFING 

F::om the beginning of this study, Atlanta officials have said that the 
classroom teacher is a key factor in whether or not students progress. As 
reported above. Area Superintendents, Resource Teachers and principals alike 
attributed the presence of red and blue signals to the quality and attitudes 
of the teaching staff. Yet neither the assignment of individual teachers 
nor the staffing of schools has been substantially affected by the intro- 
duction of information about relative performance.. Throughout the year, 
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decisions about individual teacher assignments and the staffing of entire 
schools were influenced by considerations of the racial composition of 
faculties and the declining student enrollnient. 

Information on relative performance, as reflected in the signals, was 
expected to be useful in hiring new teachers and in reassigning existing 
staff. When signals were explained and distributee to then, Personnel 
Division staff expressed interest in using signals to identify character- 
istics of teachers in red and blue signaled grades. They hoped to determine 
if training, years of experience, type of certification or teacher turnover 
appear to affect the level of performance among grades in similar schools- 
Thus, signals could be used as an output measure in a research effort to 
determine characteristics of successful teachers. The results of such an 
effort could be used as a screening device in selecting new teachers or in 
making teacher assignments. 

Although the Personnel Division staff showed an. Interest in using 
signals to Improve procedures for screening applicants, there is no evidence 
that any one follov/ed through on this interest. The small size of the staff 
of this Division in reality meant that no single staff member had th^d time 
needed to direct or conduct the research needed to answer questions about 
teacher characteristics, toreover, the value of signal information to 
Personnel staff is reduced because information about which specific grades 
and schools will require new teachers is not available during the normal time 
of recruiting and hiring new teachers. 

The 'potential Impact of signal information on the reassignment of 
existing staff is higher. As long as signals can be available prior to the 
time most new assignments are made (over the summer and during the first 
weeks of the new school year) , then signals can provide information about 
the current and proposed assignment for the teacher in question. 
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Signals played no part in the selection of new teachers, since no new 
elementary teachers havG been hired by Atlanta since the introduction of 
signals. Moreover, according to the Assis?:ant Superintendent for Personnel, 
signals played no part in central decisions about teacher reassignment made 
during the last months of the 1972-1973 school year. Area Superintendents 
provided only one instance of the use of signal information in reassigning 
staff. One Area Superintendent reassigned several teachers who had been in 
a school with several red signals to schools with several blue signals in the 
hope that the teachers would be influenced by the high performance of 
teachers in their new schools. The new assignments were to take effect in 
September 1973. 

D. IMPACT OF SIGNALS ON THE 
INSTRUCTIONAL PROGRAM 

The development of curriculum guidance for the entire school system 
and the demonstration of new instructional approaches within the school 
system are usually multi-year activities- Moreover, Title I and other 
federal programs have certain restrictions on where and how money can be 
used* Consequently, it might have been predicted that signal information 
would have little or no impact on decisions relating to the instructional 
program in the short period of this study. At best, it might have been 
predicted that signals would be used by Instruction Division staff to 
investigate the characteristics of the instructional program in grades 
where performance was either relatively high or relatively low. There is 
no evidence to show that signals were even considered in decisions involving 
the shape of the instructional prog^icam for the school system* 
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The elementary school curriculum Is being revised. As the new curriculum 
guidelines are developeci, they are being 'field-tested in a sample of schools. ■ 
The number of schools piloting the new curriculum by September 1973 had 
doubled to 20. There has been no quantitative evaluation of the new curricu- 
lum to date, nor is one planned. Neither performance information in general, 
nor the signals in particular, seem to have affected the choice of pilot 
schools or the appraisal of the curriculum being developed. 1 

The Comprehensive Instructional Program (CIP) is another example of an 
instructional activity which is aimed at improving student performance. 
When the program began in 1970, 50 of Atlanta^s elementary schools were 
designated CIP intiensive schools, on the basis of a combination of economic 
need (using 1965 data) and fourth grade reading performance. The evidence 
is that some of the 50 schools received more' attention than the other 75 
Atlanta elementary schools; some did not. In fact, as was reported in 
Chapter IV, it was left to the discretion of each Area Superintendent to 
decide how CIP Resource Teachers were to be used. The CIP conducted its own 
evaluation for the first two years of its existence, but this effort has been 
abandoned. Signals might be used as an indicator of where intensive assis tr- 
ance should be provided, and changes in signals could serve as an indicator 
of program ef f ectivei>ess. \^ile the Instruction Division staff of the CIP 
were familiar with signals, they gave no indication of having used signals 
in the operation of the program. 

Signals did not stimulate investigations of the characteristics of the 
instructional programs in red or blue signaled grades. For example. In- 
struction Division staff could cite no attempts to develop empirical evidence 
of the success or failure of particular texts. Although several principals 
reported adopting new textbooks for the- coming yeat signals played no part 
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in tlie decision rbout which texts to adopt. Nor did Resource Teachers re- 
port using signals in decisioas p.bout particular texts or materials. While 
30 percent of the first grades in one Area received red signals in tiuth and 
40 percent of the third grades in another Area received blue signals in math, 
neither Area Office personnel nor curriculum development staff reported 
having given ^ny thought to v?hat might have occurred in thos's grades. Neither 
Area Superintendents nor principals reported using signals in discussions 
about the instructional prograiii for the schools. One principal in the survey 
did use the signals to support a decision to restructure the aritlimetic pro- 
gram in a grade which had r^iceived red signals fov two years in a row. 

E. IMPACT OF SIGNALS ON EFFORTS T O 
I>jgROVE THE SKILLS OF THE 

teachii;g staff 

Two mechanisms for improving teacher skills — direct assistance to 
classroom teachers and structured teacher training activities — were described 
in the preceding chapter. The remainder of this chapter is concerned with 
whether sij^nals had an impact on the provision of direct classroom as.sistance 
to teachers. The evidence of that is largely quantitative; in the case of 
in-service training, the evidence is mostly qualitative and based on personal 
intervip>7R, 

No Atlanta official reported using signal information in the provision 
of teacher training. The Resource Teachers who organized and conducted 
workshops could cite no instances of the use of signals in decisions about 
the subjects to be offered or the location of the workshops* Neither 
principal. s nor Resource Teachers remembered directing a teacher from a red 
signaled ijrade to enroll in an in-service program or proficiency module, 
although both principals and Resource Teachers did make such recommendation 
without regard to signals. 

ERLC 



Answers to qiicst.ions about the impact of signals on Resource Teacher 
assistance to classroom teachers are more complex. As described in Chapter 
III, Resource Teachers provided information on visits to schools during the 
1972-1973 school year on the Report of School Visit form. The recorded in- 
formation was processed to create a data base for analyses of Resource 
Teacher assistance during the 1972-1973 school year, to determine if signals 
had a measurable effect on the distribution of that activity among elementary 
grades. Information was not available on the pattern of Resource Teacher 
activity prior to September 1972, nor was there any agreement a priori as 
to what should be the pattern of direct assistance by Resource Teachers, 
Consequently, the analyses have been restricted to variations within the 
pattern of activity for the 1972-73 school year* In particular, the follow- 
ing question was addressed, 

o Can variation in patterns of Resource Teacher activity during the 
1972-1973 school year be associated or contrasted with variation 
in patterns of signals for 1972? 

That is, it was assam.ed that the signals based on 1972 achievement, which 

were distributed in late 1972. could affect the pattern of Resource Teacher 

activity for the 1972-73 school year. 

By using different measures of a:itivity and different definitions of 

patterns of activity, a variety of "associations" or "contrasts" can be 

examined to provide evidence of the effect or lack of effect of signals* 

Three general measures have been used in the analysis to categorize Resource 

Teacher assistance to teachers: grades visited, time expended on visits, and 

the frequency of visits. As v/ill be seen shortly, a variety of specific 

measures can be defined fjor each of the three, T\'J0 patterns of Resource 

Teacher activity have been considered — the distribution of Resource Teacher 
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activity over all the grades that could have been visited and the distribu- 
tion of Resource Teacher activity among the grades actually visited. The. 
first distribution was used to answer the question: 

• Among the grades that existed (and conceivably could have been 
visited) did Resource Teachers give a disproportionate amount 
of assistance to grades having a particular type of signal? 

The second distribution was used to answer the question: 

• Among the grades that were in fact reported as having been 
visited, did Resource Teachers give a disproportionate amount 
of assistance to grades having a particular type of signal? 

Both questions were addressed in order to take into consideration restrictions 
on Resource Teacher activity which niay have prevented them from providing as- 
sistance to some grades, independent of the type of signal .that existed. 

All of thef analyses presented below are based on those vicits by Re- 
source Teachers which involved at least one classroom teacher in Grades 

25 

1-7 in primary or elementary schools. The focus on Grades 1-7 was dictated 
by the fact that these are the only grades for which signals existed. Thus, 
instead of the 3,181 Resource Teacher visit forms which were collected and 
used to describe Resource Teacher activities in Chapter IV, the analyses 
of the effects of signals on Resource Teacher assistance to the classroom 
are based on the 1,875 visits involving classroom teachers. The remaining 
1,306 visits were to principals and other school staff not directly 
assigned to the classroom. 

25l! The number of visits and grades used in this chapter are sometimes 
different than those used in Chapters II and IV because in those chapters 
different sets of schools were used as bases for analysis and discussion. 
Here visits to schools (or grades) for which there were no signals (blue, 
neutral or red) in 1972 were omitted. 
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The remainder of this section is divided into two parts* The first 
part examines the Impact of signals on the distribution of Ilesoi\rce Teacher 
assistance among all grades in Atlanta; the second part examines the impact 
of signals on the actual assistance provided. 

1. THE IMPACT OF SIGNAL INFOR>!ATION ON THE 
ALLOCATION OF RESOURCE TEACHER ASSISTANCE 

The results presented below compare the observed distribution of Re- 
source Teacher assistance to classroom teachers with what would have been 
expected if assistance had been given in proportion to the total number of 
all grades with each type of signal. If the observed and expected distri- 
butions do not differ significantly, then it can be concluded that the Re- 
source Teachers did not give a disproportionate amount of assistance to 
grades having a particular type of signal. Such a result could be inter- 
preted as an indication that signal information did not affect the allocation 

26 

of Resource Teacher assistance to classroom teachers. This interpretation 
seems reasonable, since the aim of the signals is to identify extremes of 
performance on the assumption that the system will react to this information 
and provide evidence of disproportionate attention to the grades exhibiting 
extremes of performance. 

Two terms are used throughout the analyses- presented below: the expected 
distribution and the observed distribution of Resource Teacher assistance. 



26. The distribution of Resource Teacher assistance in prior years may 
have "been quite different from that observed in 1972-1973. Thus, it could 
be argued that the introduction of signals was associated with a redistri- 
bution of Resource Teacher effort. Baseline data on the distribution of as- 
sistance prior to the introduction of signals, are not available, rendering 
impossible such pre/post comparisons. 
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Expected distribution . The distribution which would be expected 
if Resource Teacher assistance were provided in the same propor- 
tion as blue I. neutral and red signals^^ for April 1972 were 
distributed among the grades that existed. 

Observed distribution ^ The actual distribution of any measure 
of Resource Teacher assistance provided in 1972-73 to grades 
identified as having received a blue, neutral or red signal in 
April 1972. 

Analyses have been conducted using the observed and expected distribu- 
tions defined in terms- of seve?:al measures of Resource Teacher assistance: 
grades visited, time expended and frequency of visits. For each measure, 
the observed distribution of Resource Teacher assistance has been calculated 
for all grades in Atlanta and for all grades in each Area. The analyses 
of the distribution of Resource Teacher assistance by Area .seemed necessary 
since Resource JTeachers are assigned to an Area Office and the pattern of 
their activity might be expected to vary among the Areas. All analyses 
v/ere conducted using the observed and expected distributions defined rela- 
tive to reading and arithmetic signals. The results were similar. There- 
fore, data presented below refer primarily to reading signals. In those 
few cases where different results occur for arithmetic, the exceptions are 
noted. 



C. EFFECT OF SIGNAL INFORMATION ON 

GRADES VISITED BY RESOURCE TEACHERS 

The most basic measure of Resource Teacher assistance is "which 
grades are visited," without regard to the amount of assistance given. 
Resource Teachers state that it is not possible to visit every grade. 



27. Throughout the^e analyses, the term "blue signal" refers to 
relatively high performance as indicated by a full or half blue signal; 
"red signal" refers to relatively low performance as indicated by a full or 
half red signal; "neutral signal" refers to performance which Is not sig- 
naled as extreme. (See discussion of signals in Chapter II.) 
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A total of 806 grades existed in 1972-73 and received a signal in 
April 1972: 10.8 percent received blue signals; 81.6 percent received 
neutral signals; 7.6 percent received red signals in reading. During 1972- 
1973, Resource Teachers reported at least one visit to 485 of those grades 
(60 percent). The observed distribution of grades visited, according to 
type of reading signal vas 10.3 percent with blue signals, 83 •! percent 
with neutral signals and 6.6 percent with red signals. Thus, the observed 
distribution of grades visited is nearly identical to the existing or 
expected distribution of signals. The observed and expected distribution 
do not differ by more than 1.5 percent. 

To determine. if the differences were statistically significant, the 
two distributions were compared using a chi-square (or X^) test with two 
degrees of freedom and a sample size determined by the number of visits 
that actually occurred. The results indicate that the detected difference 
would be statistically significant only at a confidence level on the order 
of 20 percent or more. That is, the difference is not significant at any 
of the typically used levels of confidence, such as 1 percent, 5 percent 
or 10 percent* In operational terms, a 1.5 percent difference means that 
only 12 grades out of the 806 total (or 10 out of the 658 neutral grades) 
were visited that might not have been expected to be visited if assistance 
were provided proportionally. Similar results^ occur when the analyses are 
conducted on an Area basis and when arithmetic signals are used. The data 
used to obtain these results are presented in Table V-1. 
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TABLE V-1 

EXPECTED AND OBSERVED DISTRIBUTION OF RESOURCE TEACHER VISITS TO 
EXISTING GRADES, BY SIGNAL AND BY AREA, READING, 1972-73 



GRADES 
CONSIDERED 


EXPECTED , 
DISTRIBUTIONS^ 

PERCENT OF SIGNALS 
Blue Kcutral n«d 


OBSERVED , 
DISTRIBUTIONS^ 

PERCENT OF SIGNALS 
Blu€ N'eutrnl Red 


DIFFERENCE 
SIGNIFICANT AT 
LEVEL of: ^ 
1^ 5T 


ALL GRADES 


10.8 


81.6 


7.6 


10.3 


83.1 


6.6 


NO 


NO 


APXA I 


15.8 


82.5 


1.7 


13.0 


64.8 


2.2 


KO 


KO 


AREA II 


3.2 


65.4 


ItA 


3.2 


67.3 


9.5 


NO 


NO 


AREA in 


21.6 


69.0 


• 9.« 


21. A 


70.9 


7.8 


NO 


NO 


AREA IV 


3.3 


. 67.8 


■ 8.8 


3.1 


87.5 


9.4 


NO 


NO 


AREA V 

i 


11.1 


82.6 


6.1 


10.1 


85.9 


4.0 


NO 


NO 



1. Distribution that would be expected if visits were proportional to the distribution of signals. 
2» Observed distribution of Resource Teacher visits by type of signal. 

3» Statistical significance determined by a X test with tvo degrees of freedom and a aanple size equal 
to the nucbcr of. visits that occurred. 



In summary, when the number of grades visited is used as a measure of 
Resource Teacher assistance, there is no evidence to indicate that a dis- 
proportionate amount of assistance was given grades having a blue or red 
signal in reading or arithmetic. The same result is obtained when considering 
Resource Teacher visits' to all schools in the system or the schools in each 

of the five Areas. 

However, as has been pointed out, a measure based on whether or not a 

grade was visited at least once during the school year does not take into 
consideration the amount of assistance given. The number of grades visited 
is simply a measure of the minimum level of effort provided. In order to 
examine the amount of Resource Teacher assistance provided classroom teachers, 
analyses based on tima expended on Resource Teacher visits have been per- 
formed. The results of those analyses follow. 
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b. EFFECT OF SIGNAL INFORi^L\TION ON TIME ' 
EXPENDED DURING RESOURCE TEACHER VISITS 

The length of each visit reported by Resource Teachers provides a means 
of weighting the amount of assistance given in a visit. If each grade had 
only one teacher and each Resource Teacher visited only one teacher at a time, 
then the reported time could be used as a measure of activity without any am- 
biguity. However, in most Atlanta schools, several teachers are assigned to 
each grade level and, on vtiany occasions, Resource Teachers simultaneously 
visit several teachers from one or several grades. Personnel other than 
classroom teachers also participate in many visits. In order to examine the 
impact of signal information as measured by the length of Resource Teacher 
visits, a method had to be found to allocate the benefit, as measured by 
time spent on a visit, to the participants and the grades they represent. 

Since no accepted theory exists to scale the benefit of group versus 
individual meetings between teacher and Resource Teacher, two definitions 
of time were considered: 

(1) Resource Teacher Time: Each grade is allocated a proportion of 
the total time expended by the Resource Teacher equal to the 
proportion of participants who are teachers in that grade. That 
is, if a Resource Teacher spends a total of t^ minutes simul- 
taneously visiting n persons, of whom m are teachers in grade ^, 
then a measure of time expended on grade ^ during the visit v is: 

T(v,g) = 

(2) Teacher Time; Each grade is allocated a multiple of the total 
time expended by the Resource Teacher equal to the number of 
participants who are teachers in that grade-. That is, if a 
Resource Teacher spends a total of _t minutes simultaneously 
visiting n persons, of which in are teachers in grade ^, then a 
measure of time expended on grade ^ during the visit v is: 

T'(v,g) = mt. 

Resource Teacher time assumes that the benefit to each participant in 
a visit (as measured by time) is equally divided among the participants so 
that the sum of the benefit (time) equals exactly the time the Resource Teacher 

RJC 
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expended* On the other hand, the second measure, teachet tlme^ assumes that 
all participants derive the same amount of benefit as If only one had been 
present. That Is, In the second case^ benefit to each participant Is assumed 
to be independent of the number of teachers present. 

While both measures were used In the analyses performed to determine If 

i 

signal Information had an effect on Resource Teacher assistance, the results 

i 

obtained using Resource Teacher time did not differ from the results obtained 
using teacher time* Thus, while data which appear in the succeeding para- 
graphs refer only to Resource Teacher time, the findings are generally appli- 
cable to teacher time. 

28 

Resource Teachers reported spending 1,978 hours assisting classroom 
teachers of Grades 1-7. If these hours had- been expended in direct propor- 
tion to the signals assigned to the grades in reading, then 10.8 percent of 
the time would be expected to be spent in blue signaled grades, 81,6 percent 
in neutral grades and 7.6 percent in red signaled grades. In fact,. the ob- 
served distribution of Resource Teacher time was as follows: 9.6 percent in 
blue signaled grades, 82.5 percent in neutral grades, and 7.9 percent in red 
signaled grades. As with the distribution of grades visited, the observed 
distribution and the expected distribution of Resource Teacher time were very 
similar; a comparison of the difference is not significant at a confidence 
level of 25 percent or less. 

The differences between the observed and the expected distribution of 
Resource Teacher time are even less significant when viewed In light of the 
operation of the school system. For example, a difference of 1 percent be- 
tween observed and expected amounts of Resource Teacher time represents less 



28 . The total time reported by Resource Teachers, including assistance 
to principals and other staff not directly assigned to Grades 1--7 was sub- 
stantially greater than 1,978 hours. 
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than 20 hours out of the total amount reported over the entire school year. 

j 

Thus, in terms of Resource Teacher time, the sum of the absolute difference 

between observed and expected effort given to blue, neutral and red signals 

I 

ro.presents less than 47 hours of reported Resource Teachers' time over the 

i 

whole school year — ^^hardly an operationally significant difference for the 
combined efforts of the 52 Resource Teachers who submitted reports. The 
differences detected for Resource Teacher effort relative to arithmetic 
signals are even smaller than for reading. 

Different results are obtained when the distributions are calculated 
on an Area basis. In particular, the difference between the observed 
distributions of Resource Teacher time and the expected distribution of Re- 
source Teacher time relative to reading signals is statistically significant 
at the 1 percent confidence level for Areas I, III and IV. When arithmetic 
signals are used, the detected differences are statistically significant at 
the 1 percent level only in Area III. When the level of statistical con- 
fidence is 5 percent, then detected differences relative to arithmetic signals 
become significant for Area I. These results are presented in Table V-2 below. 

TABLE V-2 

EXPECTED AND OBSERVED DISTRIBUTION OF RESOURCE TEACHER TIME, 
BY AREA AND SIGNAL, 1972-73 



«. Reading Signals 



GRADES 
CONSIDERED 


expected , 
distributions! 

PERCENT OF SIGNALS 
Blue Neutral Red 


OBSERVED ^ 
DISTRIBUTIONS^ 

PERCOiT OF SIGNALS 
Blue Neutral Rcc. 


DIFFERENCE 
SIGNIFICANT AT 
LEVEL OF:-^ 
1% S7. 


ALL GRADES 


10.8 


81.6 


7.6 


9.6 


82.5 


7.9 


NO 


NO 


AREA I 


15.8 


82.5 


1.7 


9.9 


87,2 


2.9 


YES 


YES 


AREA II 


3.2 


86.4 


10. A 


1.9 


89r0 


9.0 


NO 


NO 


AREA III 


21.6 


69.0 


9. A 


35.6 


62.0 


2.4 


YES 


YES 


AREA IV 


3.3 


87.8 


8.8 


2.3 


84.7 


13.0 


YES 


YES 


AREA V 


11.1 


82.8 


6.1 


9.9 


86.9 


3.1 


NO 


NO 



(Continued on following page) 
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TABLE V-2 (continued) 



b. Arithmetic Signals 



' GRADES 
CONSIDERED 


EXPECTED 
DISTRIBUTIONS^ 

PERCENT OF SIGNALS 
Blue Neutral Red 


OBSERVED . ; 
DiSTRIBUTIONSi 

PERCENT OF bIGNALS , 
Blue Neur'-dl Red 


DIFFERENCE 
SIGNIFICANT AT 
LEVEL 0F:~ 

ir. 5r 


A2X GRADES 


8.4 


85.9 


5.7 


7.6 


86.6 


5.7 


NO 


NO 


ARZA I 


10.0 


89.2 


0.8 


6.1 


92.6 


1.4 


NO 


YES 


AREA II 


2.6 


89.6 


7.8 


1.8 


90.8 


7.4 


NO 


NO 


A51EA III 


20.5 


74.3 


5.3 


32.5 


61.2 


6.3 


YES 


YES 


AREA IV 


3.3 


87.8 


8.8 


2.3 


90.3 


7.4 


NO 


NO 


AREA V 


6.1 


89.4 


4,4 


4a 


91.3 


4.6 


vo 


NO 



1. Distribution that would be expected If visits were proportlonnl to the distribution of signfilc. 

2. Obsetv. * distribution of Resource Teacher visits by type of slfinal. 

3. Statistical slgniflcauce dctcnnlned by a test vlth tvo c'egrees of freedom and a sample sire equal 
to the nuLiber of visits that occurred. 

Vfhile the differences bet\veen the observed and the expected distribution 
of Resource Teacher time in Areas I, III and IV were statistically significant 
at the 1 percent level in reading, the differences appear to have been in- 
significant in terms of the operation of the school system. Moreover, there 
was no common pattern across Areas. In Area I, there was a slight shift in 
attention (as measured by Resource Teacher time) away from blue signaled 
grades towards grades with a neutral signal. In terms of actual Resource 
Teacher time, the maximum detected difference in Area I represented only 22 
hours of Resource Teacher effort out of a total of 368 hours reported. In 
Area III, Resource Teachers spent more time than expected with blue signaled 
grades (in reading and arithmetic) at the expense of both red and neutral 
signaled grades. The differences between observed and expected values in 
the blue signaled grades of Area III represented about 31 hours out of the 
290 hours reported by Resource Teachers in that Area. In Area IV, effort 
was shifted from neutral signaled grades to red signaled grades. The 



V-21 

differential effort on red signaled grades represented 37 hours out of the 
890 hours reported. 

Thus, except possibly in Area III, data collected on how Resource 
Teachers expended time provide no evidence to indicate that grades were 
singled out for attention according to the signal they received during the 
previous school year. This finding is particularly true when the distribution 
of time is considered relative to math signals. 

In still another effort to examine the distribution of direct assistance 
provided by Resource Teachers, a measure based on the frequency of visits was 
utilized to weight the attention given to grades visited • These analyses are 
discussed below. 

c* EFFECT OF SIGNAL INFORMATION ON THE 
FREQUENCY OF RESOURCE TEACHER VISITS 

As with the tiiae expended, a measure of Resource Teacher activity based 
on frequency of visits provides a means of weighting the amount of attention 
given to grades having different types of signals. If each grade had only 
one teacher and a Resource Teacher visited only one teacher at a time, then 
the frequency of visits to a particular grade would be equal to the number 
of visits reported. However, as has already been pointed out, most grades 
have several teachers and many visits involve several teachers from one or 
more grades. Therefore, as with the "time*' measures, it was necessary to 
adopt some mechanism to distribute each visit among all the participants. 

Once again, two measures for the frequisncy of visits to classroom 
teachers were defined. They parallel the measures defined for time spent 
assisting teachers. One measure allocates each Resource Teacher visit among 
all participants in the visit, on the assumption that the benefit to the 
participants is shared equally. The other measure assumes that the benefits 
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I 

to each participant is independent of the number of participants and, there- 
fore, allocates one visit to each participant, 

- When both measures were used to analyze Resource Teacher data, the 
results were the same regardless of how frequency of visit was defined. 
The results are also nearly identical with those reported above when time 
was used as a measure of Resource Teacher activity. In particular, when 
all grades are considered without regard to Area, the observed distribution 
of visits is quite close to the distribution which would have been expected 
If the visits had been allocated in direct proportion to the number of 
grades having blue, neutral or red signals* For example, for grades 
signaled in reading, the observed distribution of visits was 10. A percent 
to blue signaled grades, 81.6 percent to neutral signaled grades and 8.1 
percent to red signaled grades, almost identical to the expected distribu- 
tion of 10.8 percent blue, 81.6 percent neutral, and 7.6 percent red. Using 
2 

a X test of significance, the differences between these distributions are 
not significant at confidence levels up to 50 percent. The maximum dif- 
ference between the distributions occurs relative to the red signals and 
that is only a difference of 0.5 percent. This represents a total of less 
than 10 of the 1,875 visits reported by all Resource Teachers for the 
entire year. The results of the analyses of Resource Teacher frequency 
of visits may be found in Table V-3. 
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TABLE V-3 

EXPECTED AND OBSERVED DISTRIBUTION OF THE FREQUENCY OF 
RESOURCE TEACHER VISITS BY AREA AND BY SIGNAL, 1972-73 



a. Reading Signals 



GRADES 
CONSIEERED 


EXPECTED , 
DISTRIBUTIONS^ 

PERCcXT OF SIGNALS 


OBSERVED , 
DISTRIBUTIONS^. 

PERCENT OF SICICAIS 


DIFFERENCE 
SIGNIFICANT AT 
LEVEL OF:^ 




Blue 


N'eutral 




Blue 


Keutral 


Red 


IX 


51 


m GRADES 


10.8 


81.6 


7.6 


10.4 


81.6 


-8.1 


NO 


NO 


AREA I 


15.8 


82.5 


1.7 


9.8 


87.2 


3.0 


YES 


YES 


AREA II 


3.2 


86. A 


10.4 


2.1 


86.5 


11.4 


NO 


NO' 


AHilA III 


21.6 


69.0 


9.4 


36.9 


59.5 


3.6 


^"ES 


YES 


AREA IV. 


3.3 


87.8 


8.8 


2.2 


85.3 


12.5 


YES 


YES 


AREA V 


11.1 


82.8 


6.1 


9.3 


87.0 


3.7 


KO 


t:o 


b. Arithnetic Signals 


ALL GRADES 


8. A 


85.9 


5.7 . 


8,5 


86.2 


5.3 


- NO 


KO 


AREA I 


10.0 


89.2 


0.8 


5.8 


92.8 


1.5 


NO 


YES 


AREA II 


2.6 


89.6 


7.8 


2.2 


89.8 


8.0 


NO 


NO 


AREA III 


20.5 


74.3 


5.3 


34.1 


60.7 


5.1 


YES 


YES 


AREA IV 


3.3 


87.8 


8.8 


2.3 


91.2 


6.5 


NO 


YES 


ARLV V 


6.1 


89. A 


4.4 


3.4 


90*8 


5.7 


NO 


NO 



1. Distribution that would be expected if visits vcrc proportional to the distribution of signals. 

2. Observed distribution of Resource Teacher visits by type of signal. 

2 

3. Statistical aignificancc detennined by a X test with two degrees of freedca and a sample size equal 
to the nusber of visits that occurred. 
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As when time was used as a measure of Resource Teacher assistance, sta- 
tistically significant differences between the observed and expected frequency 
distributions appear in Areas I, III and IV in reading (at the 1 percent level) 
and in arithmetic in Area III (at the 1 percent level) and in Areas I and IV 
(at the 5 percent level). However, only in Area III do the differences ap- 
pear large enough to have some operational significance. 

There is no pattern to the differences observed among Areas for 
reading. In Area I, red and neutral signaled grades were visited slightly 
more often than blue signaled grades, but in operational terms, that dif- 
ference represented only about 21 visits out of 357 reported. In Area III, 
blue signaled grades were visited more often, at the expense of neutral and 
red signaled grades; the difference amounted to 57 of the 373 visits reported. 
In Area IV, red signaled grades were visited at the expense of neutral and 
blue signaled grades, but the difference represented only 29 visits out of 
793 reported. 

In summary, analyses using the distribution of visits as a measure of 
activity corroborate the results obtained when time was used as a measure of 
activity. That is, when all grades are considered, reported Resource Teacher 
activity was distributed among grades having blue, neutral and red signals 
in nearly the same proportions as the signals were distributed. And, while 
statistically significant differences between the obi^erved and proportionate 
distribution were detected in Areas I, III and IV, only in Area III were the 
differences large enough to have operational significance for the school 
system. 
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d. SUMMARY 

The preceding analyses have been addressed to the question of whether 
or not Resource Teachers gave a disproportionate amount of assistance to 
grades in 1972-73 which had received a particular type of signal in April 
1972. Resource Teacher assistance was measured in uerms of the number of 
grades visited, the amount of time spent in assisting teachers in those grades 
and the frequency of visits to those teachers. It was assumed that if 
.signal information had no effect on the allocation of Resource Teacher as- 
sistance, then the observed distribution of any measure V7onld be proportional 
to the distribution of blue, neutral and red signals among existing grades. 
The results may be summarized as follows: 

© VJTien all grades were considered without regard to Area, 
no statistically significant differences were detected 
between the observed distribution of Resource Teacher 
assistance and the expected distribution. This result 
applies when Resource Teacher assistance is measured 
by grades visited, time expended or frequency of visits. 
The result holds when either reading or arithmetic 
signals are used to define Resource Teacher assistance. 

© IThen Resource Teacher assistance was examined on an 
Area basis, rio statistically significant differences 
were detected between the observed distribution of 
grades visited arxd the expected distribution. The 
result holds when either rea-^ing or arithmetic signals 
are used to define Resource Teacher assistance. 

© When either time expended or frequency of visits was 
used as a measure of Resource Teacher assistance, 
statistically significant differences were detec\:ed 
in Areas I, III and IV. The greatest differences 
occurred relative to reading signals. Only in 
Area III, however, did the differences appear large 
enough to have any significance in terms of the opera- 
tion of the Area. 
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2. THE IMPACT OF SIGNAL INFORMATION ON 

THE RESOURCE TEACHER ASSISTANCE PROVIDED 

rhe analyses presented in the preceding section focused on the allocation 

! 

of Resource Teacher assistance. This section discusses the variation In Re- 

I 

source Teacher assistance to those grades which Xv/ere visited at all. The 
grades visited by Resource Teachers were categorized by the type of signal 
each grade received In reading in 1972. The amount of Refiource Teacher 
assistance (as measured by time expended and by frequency of visits) to blue^ 
neutral and red signaled grades was calculated. The distribuLlcn ^f Resource 
Teacher time and the frequency of Resource Teacher visits for each type of 
signal have been compared to determine whether Resource Teachers gave a dis- 
proportionate amount of assistance to either blue, neutral or red signaled 
grades. There was no way to determine in advance of the study whether 
there should be a difference in the amount of assistance provided or how 
much that difference should be. 

a. OBSERVED EFFECT OF SIGNALS IN TERMS 
OF TIME EXPENDED 

30 

Once again, Resource Teacher time was used cS a measure of Resource 
Teacher assistance. The total amount of Resource Teacher time spent in a grade 
over the entire year was determined, and cumulative irequency distributions, 
using intervals of 15 minutes in length, v/ere prepared for blue, neutral and 
red signaled grades. The three cumulative frequency distributions are pre- 
sented in Figure V-1. Each curve in Figure V-1 is presented as a smooth ap- 
proxii.iation to the step function actually calculated. 

29 . Each of the analyses discussed in this section was conducted for 
arithmetic, as well as reading. Tlie results were essentially the same for 
reading and arithmetic, and only the results for reading are presented. 

30- See p.V-17 Tor the definition of Resource Teacher time. 
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FIGURE V-1 

CUMULATIVE FREQUENCY DISTRIBUTION OF GRAdES 
ASSISTED BY PESOURCE TEACHERS: TOTAL HOURS 
OF ASSISTANCE PROVIDED DURING 1972-73 i 




HOURS OF ASSISTANCE PROVIDED 
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As can be seen from observing the three curves in Figure V-1, they 'are 
remarkably similar. In fact, using the Kolraogorov-Smirnov Two-sample Test for 
cumulative distributions, the detected differences for reading are not sig- 
nificant at the five percent confidence level when either the distributions 
arc considered for either: 

blue and neutral signaled grades, or 
blue and red signaled grades, or 
red and neutral signaled grades. 

The curves in Figure V-1 differ noticeably only when we consider grades 
In which a large amount of time was expended over the year by Resource 
Teachers. Specifically, approximately 20 percent of the red signaled grades 
visited received at least eight hours of attention but only 6 percent of blue 
signaled grades received that much assistance from Resource Teachers. These 
observations indicate that Resource Teachers favor red signaled grades when 

.:^i 

expending a relatively large amount of time.' However, the differences are 
neither statistically significant nor large enough to be operationally 
significant; 

The variation among the curves for large time values does have an 
effect on the measures of central tendency (which give weight to extreme 
values of a distribution). In particular, the mean or average time expended 
over the year in grades with each type of signal is as follows: blue sig- 
naled grades — 3,44 hours; neutral grades — 3.74 hours; and red signaled 
grades — 4»24 hours. However, using a t-test, the differences in mean time 
between red signaled grades and blue or neutral signaled grades are not 

31, Note that the -expended time is the total amount of time reported 
as being expended over the entire year by all the Resource Teachers. 
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even statistically significant at a 10 percent confidence level. The 
observed differences result from a few red signaled grades which received a 

I 

large amount of Resource Teacher; when the median is used as a measure of 
measure of central tendency, the differences are even less. In particular, 
the median amount of time expended in each category of signals is as follows: 
blue — 2.88 hours; neutral — 2.50 hours; and red — 2.92 hours. Clearly, these 
are not operationally significant differences. 

In summary, while differences can be detected in the distribution of 
Resource Teacher time spent in grades having a blue, neutral or red signal 
in reading, the differences generally are not significant and do not indicate 
a pervasive tendency to provide assistance to grades having a particular 
type of signal. 

b. OBSERVED EFFECT OF SIGNALS IN 
TERMS OF FREQUENCY OF VISITS 

The frequency of Resource Teacher visits was also used as a mea-^ 
sure of Resource Teacher assistance. The total number of Resource Teacher 
visits to each grade was calculated. Cumulative frequency distributions 
based on the total number of visits reported over the entire year were pre- 
pared for the blue, neutral and red signaled grades. The three cumulative 
frequency distributions are presented in Figure V-2. Once again, the curves 
appear as smooth approximations of the step functions actually calculated. 

As can be observed, the three distributions tend to behave similarly. 
In fact, using the Kalmogorov-Smirnov Two-sample Test, the distributions of 
the frequency of visit measure are not statistically significant when con- 
sidering the distributions for reading relative to: 

32. See p,V-21 for the definition of frequency of visit. 
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FIGURE V-2' 

CUMULATIVE FREQUENCY DISTRIBUTION OF GRADES 
ASSISTED BY RESOURCE TEACHERS: TOTAL NUMBER 
DF VISITS OCCURRING DURING 1972-73 




.5 I 2 • • ' 7 ' ' ' ^2 ' • • >16 



NUMBER OF VISITS 



fjOTEJ THE HORIZONTAL AXIS—THE NUMBER OF VISITS— HAS BEEN 
ELONGATED BETWEr.N 0 AND 2 TO DISPLAY THE CUMULATIVE EFFECT 
OF VISITS THAT iiiVOLVE SEVERAL CLASSROOn TEACHERS. THAT 
IS^ EACH Cr^ADC RECEIVES A PROPOi^TIONAL'CREDlT FOR THE VISIT 
EOUAL TO THE PROPORTION OF TEACHERS FROM THAT GRADE WHO 
WERE INVOLVED IN THE VISIT. (SEG PA6EV-21fOR A COMPLETE 
DEFINITION OF RESOURCE TEACHER FREQUENCY OF VISIT KEASURES.) 
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blue and neutral signaled grades, 
blue and red signaled grades, and 
red and neutral signaled grades. 

As with. the distributions of time expended, the curve for the dis- 
tribution of visits for grades vhlc.h had a red reading signal dominates the 
other two curves for small values of the measure; the reverse is true for 
grades which received a relatively large number of Resource Teacher visits. 
However, the difference between the curves for large values of the "fre- 
quency of visit measure*^ is not as great as when "time" was used as a 
measure of activity. Thus, the means or averages for the distributions in 
Figure V-2 do not differ from one another as much as when time was the 
measure of activity. In particular, the mean or average number of visits 
that occurred to grades having blue signals was It 9 visits; for neutral 
signals, 2.0 visits; and for red signals, 2.6 visits. The medians for the 
three distributions are- 2.7, 2.3 and 2,8 visits respectively. Clearly, the 
differences that exist between these measures of central tendency of the 
distributions of Resource Teacher effort among grades having blue, neutral 
or red signals does not have much significance in terms of the operation of 
the school system. 

3. SUMMARY 

The analyses of the characteristics of the Resource Teacher visits 
which were reported In 1972-73 failed to show any operationally Important 
tendency for Resource Teachers to single out either blue, neutral or red 
signaled grades for special attention. Analyses have also been conducted 
on a grade level basis as well as on an Area basis. In a few cases, sta- 
tistically significant differences were detected between the observed and 
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expected distributions, but none of the differences have obvious operational 
significance nor are they as great as those detected in the Area analyses. 
Furthermore, there is no apparent pattern of differential assistance to a 
particular grade or group of grades having a particular type of signal. 
For these reasons, the detailed discussions of the grade-by-grade analysis 
have been omitted from this report. 

The results and implications of the findings on the effect of signals 
that have been reported in this chapter are further described in Chapter I. 
Chapter I also presents other principal conclusions and implications of 
the study. 



